Видео

is there any SEO expert that isn't overly excited when speaking? someone dark maybe? a bit punk

Short and to the point, and most importantly honest. Subscribed

very cool. really liked the video !

Thx for great video! so what is the better way to teach a model new knowledge, if FT is somehow only good for structure? thx much!

Nice video.

good one, thank you

doing a f'cking god's work

Thank you for the explanation, It helps me a lot.

how is Lora fine-tuning track changes from creating two decomposition matrix? How the ΔW is determined?

Short videos are great

@15:28, there is nothing great about adding the extra LoRA parameters to the weights that makes it easier to swap the behaviour of the model at inference time because the difference between adding the matrices and loading the entirely new weight matrix from different finetuned models to the model architecture is negligible.

My left ear enjoys this video

作为初学者，我想请问一下如何运行本篇文章的代码，这个复杂吗，谢谢你❤

Amazing explanation, Thank you!

@AlexG4MES 8 months ago This was beautifully explained. As someone who relies exclusively on self learning from online materials, the mathematical barriers of differig notations and overly complex wording is the most time consuming challenge. Thank you for such a distilled explanation, with only the notation and wording that makes sense for an intuitive and initial dive understanding. Subscribed!

I want to ask these template line we have to write

Outdated, not working

Hello ... new to the field of llm training .. thanks for putting up these two videos .. However , I am a bit confused by the comment ' ...you can do it during inference time' ? As per my understanding the weight updates are done during fine tuning ... and they are later used during inference ... if the task is changed we just revert back to the pertained weight by getting rid off the weight update for current task .. and fine tune the model based on the new task to get the new weight update ... the new update are then again used later during inference .. so the weight updates are during fine tuning only ... which I think why the authors mentioned that batch processing is not obtained by LoRA (base) ... though possible and difficult ... may be there is some future version where its implemented ? I am not sure .... but please correct me if I am conceptually wrong ...

why are u ourple

What's the difference between less parameters, and low intrinsic weights? because weights are parameters of Neural Net isn't it?

The one issue I have had is that this causes memory footprint to grow. But it sounds like you should be able to merge it into the base model at the end to keep the same footprint. Maybe that is something for me to try. I wonder if this low rank decomposition can be used for model distillation. Instead of just quantizing weights.

Man that video is fireee! Thank you for your work!

Thanks a lot for this amazing explanation. I am fine tuning Mixtral 8x7B and using QLoRA have been able to perform test runs on Colab Pro using A100 machine.

Thanks for the great intro to LoRA. I liked your graphics and your take-aways, also you energetic presentation :)

How to use it locally through gpt4all? Thank you!

Thanks a lot for this video! This is the first time I see a good explanation on this LoRA thing! 14:45 One minor note, is that it would indicate that the model has a low intrinsic info only if you could get rid of the original weights and just stick to the lora. That is, during the lora finetune training, if you could get away with while decaying the original (non-lora) weights down to zero. So I think that what has a low intrinsic info is "what you have to change from the base model" for your finetuning needs - but not the base model itself.

Probably a case where overfitting can be beneficial. 😁

Any plans to continue the series? Sounds like exactly what I'm looking for.

Do I need to "download" LLM?

Attempting to run the notbook but I keep getting ValueError: Attempting to unscale FP16 gradients. Tried different colab envs but no luck.

Great video. Wish you showed the comparison against the base model. Just to clarify, we are not able to use the LORA model generated from model A with a different base model?

Christ, amazing job! please keep going!

Impresive, you rock, waiting for more videos

Getting "ValueError: Attempting to unscale FP16 gradients." when running the cell with trainer.train(). Any idea?

sudo docker run --gpus=all --shm-size 64g -p 7860:7860 -v ${HOME}/.cache/root/.cache --rm alpaca-lora-demo <- no errors, no output, server localhost doesnt work ;(

this is a masterpiece

Do you know how much GPU RAM the meta-llama/Llama-2-70b-chat model would take to fine-tune?

I particularly appreciate the depth of research and preparation that clearly goes into this video. It's evident that you're passionate about the topics you cover, and your enthusiasm is contagious. Your dedication to providing accurate information while maintaining an accessible and entertaining format is commendable.

Thank you so much! I'm working on a project to develop a chatbot for student advisory services and i am contemplating between two approaches: Fine-Tuning and Retrieval-Augmented Generation (RAG). Here are the key tasks the chatbot needs to address: Answering general queries about university courses and requirements. Providing personalized advice on study plans and schedules. Assisting with administrative processes (enrollment, documentation, etc.). Offering support for common academic and personal challenges faced by students. Given these responsibilities, which approach would be more suitable? Fine-Tuning could offer more precise, tailored responses, but RAG might better handle diverse, real-time queries with its information retrieval capabilities. Any insights or experiences with similar projects would be greatly appreciated!

I like the videos and I'm trying to learn the whole AI thing which is no doubt a lot to grasp when you're coming into it new. the only thing that I will change is, if possible just let people know in the beginning that you will be explaining further details of certain things throughout the classes and then let it be. Because what I'm finding is that as you're explaining it, when I start to follow you and I'm synced into what you're saying you kind of break that concentration by repeating the fact that you're going to explain the details again later. if that makes any sense hopefully you understand what I'm saying. Because I am still so new at this if you have any other information that is more up to date I would greatly appreciate anything that you can help me along with. All in all I like the videos.

Why does deltaW need to be represented by both WA x WB? Why couldn't it be represented using just smaller matrix?

When I run inpaintanything in StableDiffusionUI, especially when I run inpainting, I keep getting error Unexpected end of JSON input.I ran it through Google Labs, what should I do?

Отличный туториал)

any chance you can make an example of fine tuning code llama like this

Could you explain why this saves memory? Don't you need the pre-trained weights in backprop to calculate the difference matrixes and during the forward pass?

I am able to inference the fine tuned model from lit llama but I want to do the conversation with the fine tuned model. How can I do it with the code in repository.

Thank you for the good explanation of LoRA! Super!

Great video! What wpuld be different if i do download the model not on colab but locally? Which lines do change in the code?

Dude. This video is amazing

Hi a question, can we use lora to just reduce the size of a model and run inference, or we have to train it always?