I looked at many blogs and explanations but none of them got to the practical usage of LoRA and showed exactly how it is used in practice. This is exactly what I was looking for.
I'm genuinely impressed by the content and presentation you've crafted for the ML/AI community. The way you've structured the presentation is both user-friendly and cohesive, allowing for a gradual and understandable flow of information.
Can you do a video on finetuning a multimodal LLM (Video-LlaMA, LLaVA, or CLIP) with a custom multimodal dataset containing images and texts for relation extraction or a specific task? Can you do it using open-source multimodal LLM and multimodal datasets like video-llama or else so anyone can further their experiments with the help of your tutorial. Can you also talk about how we can boost the performance of the fine-tuned modal using prompt tuning in the same video?
Thank you for the great video. Now I am wondering how LLM or diffusion trined with LoRA. These models have many layers which attention, dense, fully connected.. how does LoRA adapted for this? In the digit example, number '9' becomes better result after LoRA adaption. But other numbers accuracy become much worse. Is this natural for LoRA apdation? or We can make all number accurately with LoRA (which is trained for 9)? Thank you very much!
🎉Top tier content!, thank you, I was looking at the net results for the other digits in your demo and realized they were worse off, then thought about it a bit more deeply, it looks like you trained a single B and A matrix and added to all layers, where I think an improvement would be a separate BA matrix for each layer. Curious your thoughts on this?
Hi @tljstewart Actually, in my code we train 3 different pairs of A and B, one for each of the layers. That's why I call the method "register_parameterization" method 3 times, one for each of the layers. Each A and B matrix has a different dimensions, because the dimensions of each layers are different. Usually we can't know which layer we should fine tune or not, unless we have a clue on what each layer may be doing (this can be said only for very specific architectures like the Transformer).
Ah thanks @umarjamilai I reviewed the code again, and it appears you do freeze the original model and train a Lora matrix for each layer. That leads me to a couple questions, how do you save the Lora weights and then how would you load them back in, for sharing say on hugging face? Just for an example, how might you load stable diffusion then load a Lora programmatically?
i think you should save the lora paramters beside the orginal model, and during inference stage, you can enable, or even add different lora to get the fine tuned result@@tljstewart
Thank you for the very cool video! Can you suggest any ways that we can use to combine the finetuned and the pretrained models so they can perform well on all digits?
To push loRa to its efficient limit, does it make sense to find the rank of the original weight matrices by finding statistically significant singular values with Marchenko-Pastur Law to choose the rank of the LoRA matrices?
Is it like the base model is stored in 4bit and as the data (X vector) passes through the layer that layer is first dequantized and then the matrix multiplication is done (X*W)? And the same thing for LoRA as well? and after we get Y (by adding output of lora and base layer) the W and LoRA layers are again quantized back to 4bit? and Y is passed on to next layer? Also, if the LoRA is at the base of the model, does that mean to update the parameters of this LoRA we need to calculate the gradients of loss wrt all the W and LoRA matrices above it?
Hello everyone, I have been in Deep Learning space for about some time now..Sometimes when I come across new codes like this in the video..I just can't keep up many of things in code feels new and gets overwhelming to understand How do I bridge this gap effectively?
In LoRA, Wupdated = Wo + BA, where B and A are decomposed matrices with low ranks, so i wanted to ask you that what does the parameters of B and A represent like are they both the parameters of pre trained model, or both are the parameters of target dataset, or else one (B) represents pre-trained model parameters and the other (A) represents target dataset parameters, please answer as soon as possible
Thanks for you great work! May I ask a question? When Lora is enabled, the accuracy is 84.3% which is much worse than the original accuracy. So is it really beneficial when we enable Lora?
Of course the accuracy may degrade depending on the rank of the LoRA matrices, because the model has less parameters and so, less degrees of freedom. But it's not a rule: an overparameterized model may not suffer at all from degradation when using LoRA.
Cool video mainly due to the topic. Sometimes, I had to rewind backwards, bacuase I could not get something, mainly why the reduction rank was 2 - is this just a chosen parameter?
Hi! The rank of the matrix is a hyper-parameter and in my PyTorch implementation, I had chosen a rank of 1. The lower the parameter, the lower the size of the matrix, but also the higher the loss of "precision", because the matrix may have an intrinsic dimension higher than the chosen hyper-parameter. If it doesn't make sense to you, I suggest you read what is the rank of the matrix and how dimentionality reduction works in PCA. That should give you the math background.
As of now, LoRA is used for fine-tuning. For reducing the "size" of the model, there are quantization techniques. I'll make a video about them in the future. Have a nice day!
@@umarjamilai ok i just thought about it and uhhh, yeah, i dont see how, i had a misconception in my head, i forgot that the input data goes through the weights one layer at a time hence the output of layer 1 is the input of layer 2, plus they have activations functions that might make the process non-linear, my bad, have a nice day.
Hi! Putting the AB matrix on the GPU while the rest of the model on the CPU still has one problem: the loss. I have never tried it, but I believe PyTorch would complain when it tries to compute the loss (which involves both the frozen weights and the AB matrix). You can try using my notebook (freely available on my GitHub) and comment with the result of the experiment :D
Thanks for the reply. So in Lora also we need to store the pretrained weights in the GPU. Also can you make a detailed video on Flash Attention and Retentive Transformer.
@@weiyaoli6977 That's the number of parameters due to LoRA, which is the size of the two matrices. When you save the model, you save the two matrices separately, so you only need to consider the size of each separately and sum them together. When you use LoRA, on the other hand, you need to multiply the two matrices.
![Natanael Cano - Amor Eterno [Official Video]](http://i.ytimg.com/vi/EO4prCs6WpM/mqdefault.jpg)
As usual the full code and slides are available on my GitHub: github.com/hkproj/pytorch-lora
🥰🥰
thank you 感谢你的分享！
thanks a lot
I looked at many blogs and explanations but none of them got to the practical usage of LoRA and showed exactly how it is used in practice. This is exactly what I was looking for.
Perfect.
Genius.
Simple.
To the point.
Theoretical.
Practical.
As always, one of the best explainer on RUclips
One of the best explanation out there. Also the hands on code piece was just awesome!
I'm genuinely impressed by the content and presentation you've crafted for the ML/AI community. The way you've structured the presentation is both user-friendly and cohesive, allowing for a gradual and understandable flow of information.
I have to say one of the best youtube channels out there. And thanks for sharing the code!
best explanation about lora, thank you...
Thank you so much for your explanations, they are helping me out tremendously during my master thesis work!
Rock solid content once again. From scratch implementations are soo beneficial.
Umar thank you for the content, really helps to grasp what the concepts are .
simple use case and clear explanation thanks for this please do more of this like implementing from scratch videos
This channel is the best, 😊❤
LOVE YOUR VIDEOS LOVE YOUR KITTY'S VOICE! I MISS MY KITTY YOUR KITTEN EXACTLY MEOWS LIKE MINE WHEN I USED TO RECORD!!!! Thanks🖤👑🖤.
😸😸😸
Very good and clear explanation, thank you!
this was flawless. FLAWLESS!
Another awesome video, you're a gem. Thank you for your work, do keep making these kind of videos on the latest research papers.
Amazing video, everything was well explained, Is just what I was looking for, explanations and coding, thank you so much!
Loved your explanation
This is such quality content! Thank you!
this is the best video on LoRA.
Your videos are pure gold
Amazing explanation! Thank you.
This was insanely clear!
Such a great RUclips channel. Keep the great work!!!
so amazing tutorial
perfect, this really helps me a lot
Very good explanation. Thank you!
Great video, really impressed by the video and channel, deservers a like.
Please do a video where you show the process from scratch so we can do this with voice models ✊🏼
I was going insane until I came across this amazing LoRA video ,an oasis for me, Can you also explain QLoRA?
That Great. Thank You. You are the god!!
Amazing Explanation.
Good explaining. 👍👍
Great job!
Was hoping you would combine LORA in your stable diff video.
Thanks!
thank you :)
22:57 Why other digits accuracy can be decrease? I dont understand here.
Great video! Would you consider doing a tutorial on finetuning llama2 7b model using lora?
So you assume there is no activation function used neither in the layers contained in the frozen W layer nor in the lower representation AB layer?
Can you do a video on finetuning a multimodal LLM (Video-LlaMA, LLaVA, or CLIP) with a custom multimodal dataset containing images and texts for relation extraction or a specific task? Can you do it using open-source multimodal LLM and multimodal datasets like video-llama or else so anyone can further their experiments with the help of your tutorial. Can you also talk about how we can boost the performance of the fine-tuned modal using prompt tuning in the same video?
Thanks!
Thank you for the great video.
Now I am wondering how LLM or diffusion trined with LoRA.
These models have many layers which attention, dense, fully connected.. how does LoRA adapted for this?
In the digit example, number '9' becomes better result after LoRA adaption.
But other numbers accuracy become much worse.
Is this natural for LoRA apdation? or We can make all number accurately with LoRA (which is trained for 9)?
Thank you very much!
🎉Top tier content!, thank you, I was looking at the net results for the other digits in your demo and realized they were worse off, then thought about it a bit more deeply, it looks like you trained a single B and A matrix and added to all layers, where I think an improvement would be a separate BA matrix for each layer. Curious your thoughts on this?
Hi @tljstewart
Actually, in my code we train 3 different pairs of A and B, one for each of the layers. That's why I call the method "register_parameterization" method 3 times, one for each of the layers. Each A and B matrix has a different dimensions, because the dimensions of each layers are different.
Usually we can't know which layer we should fine tune or not, unless we have a clue on what each layer may be doing (this can be said only for very specific architectures like the Transformer).
Ah thanks @umarjamilai I reviewed the code again, and it appears you do freeze the original model and train a Lora matrix for each layer. That leads me to a couple questions, how do you save the Lora weights and then how would you load them back in, for sharing say on hugging face? Just for an example, how might you load stable diffusion then load a Lora programmatically?
i think you should save the lora paramters beside the orginal model, and during inference stage, you can enable, or even add different lora to get the fine tuned result@@tljstewart
@umarjamilai ^^
Great Presentation. I just want to know are there open source libraries for LLM models so that I can fine tune them.
How to load and save the lora weights can anyone explain?
Thank you for the very cool video! Can you suggest any ways that we can use to combine the finetuned and the pretrained models so they can perform well on all digits?
To push loRa to its efficient limit, does it make sense to find the rank of the original weight matrices by finding statistically significant singular values with Marchenko-Pastur Law to choose the rank of the LoRA matrices?
so while inferencing, we ll only use a and b? but arent a and b matrices used to handle only digit 9?
Is it like the base model is stored in 4bit and as the data (X vector) passes through the layer that layer is first dequantized and then the matrix multiplication is done (X*W)? And the same thing for LoRA as well? and after we get Y (by adding output of lora and base layer) the W and LoRA layers are again quantized back to 4bit? and Y is passed on to next layer?
Also, if the LoRA is at the base of the model, does that mean to update the parameters of this LoRA we need to calculate the gradients of loss wrt all the W and LoRA matrices above it?
Hello everyone, I have been in Deep Learning space for about some time now..Sometimes when I come across new codes like this in the video..I just can't keep up many of things in code feels new and gets overwhelming to understand
How do I bridge this gap effectively?
In LoRA, Wupdated = Wo + BA, where B and A are decomposed matrices with low ranks, so i wanted to ask you that what does the parameters of B and A represent like are they both the parameters of pre trained model, or both are the parameters of target dataset, or else one (B) represents pre-trained model parameters and the other (A) represents target dataset parameters, please answer as soon as possible
Thanks for you great work! May I ask a question? When Lora is enabled, the accuracy is 84.3% which is much worse than the original accuracy. So is it really beneficial when we enable Lora?
Of course the accuracy may degrade depending on the rank of the LoRA matrices, because the model has less parameters and so, less degrees of freedom. But it's not a rule: an overparameterized model may not suffer at all from degradation when using LoRA.
Cool video mainly due to the topic. Sometimes, I had to rewind backwards, bacuase I could not get something, mainly why the reduction rank was 2 - is this just a chosen parameter?
Hi! The rank of the matrix is a hyper-parameter and in my PyTorch implementation, I had chosen a rank of 1. The lower the parameter, the lower the size of the matrix, but also the higher the loss of "precision", because the matrix may have an intrinsic dimension higher than the chosen hyper-parameter. If it doesn't make sense to you, I suggest you read what is the rank of the matrix and how dimentionality reduction works in PCA. That should give you the math background.
Hi Umair ,is instruction fine tuning and full fine tuning both are same
Why are two matrices, A&B needed instead of just one?
Because you the multiplication of the two matrices produces the original one.
15:30 🤣🤣The comment is hilarious, rich boy net
Hi a question, can we use lora to just reduce the size of a model and run inference, or we have to always do the fintuning
?
As of now, LoRA is used for fine-tuning. For reducing the "size" of the model, there are quantization techniques. I'll make a video about them in the future.
Have a nice day!
why dont they lora the entire model's weights both the original and the changes ?
How would you LoRA the original weights?
@@umarjamilai ok i just thought about it and uhhh, yeah, i dont see how, i had a misconception in my head, i forgot that the input data goes through the weights one layer at a time hence the output of layer 1 is the input of layer 2, plus they have activations functions that might make the process non-linear, my bad, have a nice day.
For fine-tuning, I have a question suppose we store the pre-train matrix in a cpu and load the AB matrix in the gpu for fine-tuning. Will this work?
Hi! Putting the AB matrix on the GPU while the rest of the model on the CPU still has one problem: the loss. I have never tried it, but I believe PyTorch would complain when it tries to compute the loss (which involves both the frozen weights and the AB matrix). You can try using my notebook (freely available on my GitHub) and comment with the result of the experiment :D
Thanks for the reply. So in Lora also we need to store the pretrained weights in the GPU.
Also can you make a detailed video on Flash Attention and Retentive Transformer.
why b + a not b * a
Where did you read B + A? 🤔
d=1000, k=5000, p=5000(original). lora: 1000*1+1*5000=6000. so from the formula it is A*B. why A+B here? Thanks@@umarjamilai
@@weiyaoli6977 That's the number of parameters due to LoRA, which is the size of the two matrices. When you save the model, you save the two matrices separately, so you only need to consider the size of each separately and sum them together. When you use LoRA, on the other hand, you need to multiply the two matrices.