but i guess here's one: finetuning qwen2.5-32b on various ai related github prjs that are written in python, with a focus on agentic workflows, maybe individual finetunes for each agent python lib / framework, such as autogen, langchain / langgraph, ...
Sadly this may be the first time I disagree with that sentiment. Haha. I loved it but was looking for a course link at the end because it was too much too fast ;)
@@OilersFlash Yea bro (actually uncle, cuz I'm 15 and your dp shows...), I realized that the viewer will need some pre knowledge of LLMs and its workings and also some pre basic knowledge of fine-tuning! But yea the video was good actually!
I appreciate that this content is geared towards those with prior knowledge, but I think it would be really beneficial to create a beginner-friendly video that breaks down the process into clear, step-by-step instructions. That way, newbies like me can easily follow along and learn from your expertise.
This is exactly how we are tuning our open source LLMs, the use of unsloth+LORA is key. Validation of the learning is an adventure. We did this over the same tech stack mentioned here. Very good video, of course, lots of details behind each step that cannot be explained in a short video. Great use of lighting and your pace is excellent. Looking forward to more vids.
it depends, this is useful, just if you already know how to do it and you just need a refresh on the steps. In other words, you want to watch it just if you do not need it
What a great tutorial, so much info in 5 mins. It took me a while to fill in the gaps because I didn't even know what Jupyter notebook was but I now have this running on my local GPU. You can (just) run this example with 16GB vram.
I was already taken by the clear content but the KEY highlight of simply mentioning Conda shows the high quality of your informations. Subscribed and thanks for the tuto.
Every how-to I've found so far glazes over the training datasets. Like here she goes from here is 100k sql dataset, then here are is how the model expects the prompt, but then doesn't show the format of how the dataset is loaded into the model. What does the sql_context look like, what does the sql_prompt look like, what is the sql, and sql_explaination? Fine tuning is one thing but people also need to know how to build and integrate datasets for training.
@warpdotdev In the interest of establishing the repeatability of your result can you indicate whether the output @ 5:01 is unedited? typical? Mine is amending the prompt and typically produces output in the format of the Alpaca training text generating entries of 'SQL Prompt', 'SQL', 'Explanation', 'Question', 'Instruction', 'Company database' in the output.
Great video straight to the point, but could you please elaborate more on feeding custom dataset to the script? What are the steps? You wrote it right on the script or did you load it from a different file? Or did you upload to hugginface and use the token from there? Im confused
Thank you for your informative tutorial. I’ve installed Ollama on Windows 10 and currently only use it for interactive conversations through CMD. However, I asked a friend, and they mentioned that Ollama cannot be used for fine-tuning. Why is that? Additionally, Unsloth offers convenient online fine-tuning, but if the training data requires privacy and fine-tuning needs to be done locally or even if the base model is in Safetensors format from others-where should I start? Are there any learning resources or references you would recommend? Thank you!
I'm working on a RAG application. I'm using a pdf file as a text data and I have clean the data as well using NLTK. I already have ollama install in my local system and also llama2 model. I embed the data in to vector form using ollama embedded model mxbai-embed-large and store it on chromabd. but if I give the prompt and get response, it give me the response but not according to my data. and how do I finetune the model? I just need guidance and sequence so I work on the project. It's important to me. I need your guidance. please reply. I'm using windows10. Thank you!
@@ibrahimgamal7603 Could be anything such as you are not using a good doc extractor or your embedding model sucks. Changing chunk and overlap size might help. Don't go for finetuning straight away, do some experimentation and figure out where the problem lies exactly otherwise even finetuning won't work.
Hey, I have a question. I'm currently using gpt api tokens to generate models response thru python and run it over and over. Models are using information from knowledge base folder -- it would be easier to manage it through llama? My ideal scenario is to generate content model, by model and then use that output to generate complete story. Which solutions will you recommend for that purpose? Thanks
Thank you for video. I followed your approach for finetuning the model text to SQL. When I work on my own database, its performance is not good. Unable to generate SQL query. I even used agent and examples queries our database. I am requesting suggestion from you. Thank you
I would appreciate if this tutorial went more into detail on how to make a dataset comply with the model's promt template. From this video, there is no data processing being done and i think in real cases you would have to work with the data, or write a script, that chnges the data so that it fits with the prompt template (e.g. the Alpaca one she shows in the video)
if I train a model, will the responses be restricted to just my training data/examples? Or does the model's original training also play a role? For example, say I trained it on model to respond with vehicle specs. I feed it a vehicle name and the model returns the vehicle specs. Say I finetune the model with 50 examples (honda civic, toyota camry, volvo xc90, etc).... however, in reality there are 300,000 possible vehicles. Would finetuning on 50 examples help? Sure, I can put the 50 models in the prompt but then I'd just be wasting tokens.
Can you train 12B on 24GB or is 12B too big? Another question is if you have multi-turn data (conversations) can you finetune on that? The examples I see are for Q:A pairs.
Great tutorial! Thanks. Really like the definitions and explanations rather than just glossing over the code.. would love to see a full tutorial series 😮 I'd like to try to code it myself, but it would be good if you could share the code too.
Where is the data? I did not see any data given to the model to use. I feel like its perfectly possible to have a model "train in realt time" where you tell a model go in to learning mode. Then it will "learn" or "memorize" data you give it. Specifically text based data. And then it has a chance of using it when you ask a question related to that data. Why has this not been done yet? Or has it? I guess what im talking about is somthing like custom GPTs. Which i feel would be eaiser and more worth it. Its just im looking for that but on device.
that subselect that it generated @5:01 makes me instinctively cringe - hoping it would generate the subselect as a join... - for instance why do: "select p.* from posts as p inner join posttags as pt on p.post_id = pt.post_id where tag_id in ( select tag_id from tags where name = 'terminal' )" instead of: select p.* from posts p inner join posttags pt on p.post_i = pt.post_id inner join tags t on t.tag_id = pt.tag_id where t.name = 'terminal' seriously - the 2nd one appears easier to optimize. right? oh well....
It's a great how to but you missed a "before trainng" and "after training" examples of how the model responded. Good video anyway. Couldn't get much out of it though
Here it is, she says.. For simplicity.... A pip install command.. In a video.. With no pasted text to copy in the description. That's the opposite of simplicity.
Thanks! Although I do have a question why you show so much of yourself... I admit it's nice and pleasant to look at, however I would prefer to look at the code or things you talk about. Especially when I watch your videos at work I would definitely like to avoid someone thinking that I watch some silly videos during work while I actually watch work related stuff. Anyway, really interesting videos so thanks again! 😊✌
why didn't you directly use hugginface trainer to train, why using unsloth? I want to know what was the benefit of using unsloth over hugginface trainer.
You move and changing Screen very fast we understand nothing and Result and where file store and after that you merge with Ollama model nothing understand.
Why are there so many videos that are useless as tutorials and give the impression that they are only about self-promotion? Seriously, anyone who understands this quick run-through here doesn't need any more tutorials, and for the vast majority of the rest, it's probably pretty useless.
Good video, but it’s hard to imagine that you you actually sat there and edited out every breath and pause so that the whole video would sound like one massive run on sentence, only to shave off some 20 or 30 seconds of duration.
![Noob To Max With DRAGON REWORK In Blox Fruits [FULL MOVIE]](http://i.ytimg.com/vi/LnBMOinoOvA/mqdefault.jpg)
What data would you fine tune your LLM on?
what is your hardware ? are you using intel or amd threadripper ?
that depends on the use case... are you asking for what use cases people are aiming to support?
but i guess here's one: finetuning qwen2.5-32b on various ai related github prjs that are written in python, with a focus on agentic workflows, maybe individual finetunes for each agent python lib / framework, such as autogen, langchain / langgraph, ...
youtube need more chanels alike this. great job
I'm using it to fine-tune a Minecraft bot (Mindcraft from Emergent garden)
No over rating, no over talking, streight forward, love it.
Sadly this may be the first time I disagree with that sentiment. Haha. I loved it but was looking for a course link at the end because it was too much too fast ;)
@@OilersFlash Yea bro (actually uncle, cuz I'm 15 and your dp shows...), I realized that the viewer will need some pre knowledge of LLMs and its workings and also some pre basic knowledge of fine-tuning! But yea the video was good actually!
@@siddhubhai2508 it is good ;)
I appreciate that this content is geared towards those with prior knowledge, but I think it would be really beneficial to create a beginner-friendly video that breaks down the process into clear, step-by-step instructions. That way, newbies like me can easily follow along and learn from your expertise.
This is exactly how we are tuning our open source LLMs, the use of unsloth+LORA is key. Validation of the learning is an adventure. We did this over the same tech stack mentioned here. Very good video, of course, lots of details behind each step that cannot be explained in a short video. Great use of lighting and your pace is excellent. Looking forward to more vids.
Thank you straight to the point
I usually have some ptsd when looking at tutorials
it depends, this is useful, just if you already know how to do it and you just need a refresh on the steps. In other words, you want to watch it just if you do not need it
What a great tutorial, so much info in 5 mins. It took me a while to fill in the gaps because I didn't even know what Jupyter notebook was but I now have this running on my local GPU. You can (just) run this example with 16GB vram.
amazing quality of editing, sound, video - beyond the programming side!
A pleasure for the eyes and ears to watch!
Great video for people who know coding and local llm but havent finetuned!
Knowing such a deep technical subject is one thing, but teaching it well is another! well done young lady.
I was already taken by the clear content but the KEY highlight of simply mentioning Conda shows the high quality of your informations. Subscribed and thanks for the tuto.
Easiest subscribe of my life, just wanted something easy and straight to the point
Insanely good video!! Straight to the point and great presentation
This is what I am thinking to do. Finding this video maybe saved half a day
Five minutes made useful. Thank you for the crisp and neat video. ❤
Every how-to I've found so far glazes over the training datasets. Like here she goes from here is 100k sql dataset, then here are is how the model expects the prompt, but then doesn't show the format of how the dataset is loaded into the model. What does the sql_context look like, what does the sql_prompt look like, what is the sql, and sql_explaination? Fine tuning is one thing but people also need to know how to build and integrate datasets for training.
Agreed. Every video is skipping over detail to actually build your own functionality.
That’s kind of the point. It’s open source open secret
@warpdotdev
In the interest of establishing the repeatability of your result can you indicate whether the output @ 5:01 is unedited? typical? Mine is amending the prompt and typically produces output in the format of the Alpaca training text generating entries of 'SQL Prompt', 'SQL', 'Explanation', 'Question', 'Instruction', 'Company database' in the output.
Yes, same. Would be useful if they share the link to the code.
Great explanation and presentation of LLM.
Great video straight to the point, but could you please elaborate more on feeding custom dataset to the script? What are the steps? You wrote it right on the script or did you load it from a different file? Or did you upload to hugginface and use the token from there? Im confused
Hi there.
It's wonderful.. Will you Please share the notebook and also Google Collab notebook?
Immediately subscribed
great and short video guys incredible!!
I would really appreciate a version of this video but using a Mac thank you so much
What a clean presentation.
at 2:00 is 8 Billion not 8 bit.
Great video. To the point. Effective
nice and straight forward approach
Appreciate the brevity. Thank you.
Thank you for your informative tutorial.
I’ve installed Ollama on Windows 10 and currently only use it for interactive conversations through CMD.
However, I asked a friend, and they mentioned that Ollama cannot be used for fine-tuning. Why is that?
Additionally, Unsloth offers convenient online fine-tuning, but if the training data requires privacy and fine-tuning needs to be done locally
or even if the base model is in Safetensors format from others-where should I start?
Are there any learning resources or references you would recommend?
Thank you!
I'm working on a RAG application. I'm using a pdf file as a text data and I have clean the data as well using NLTK. I already have ollama install in my local system and also llama2 model. I
embed the data in to vector form using ollama embedded model mxbai-embed-large and store it on chromabd. but if I give the prompt and get response, it give me the response but not according to my data. and how do I finetune the model? I just need guidance and sequence so I work on the project. It's important to me. I need your guidance. please reply. I'm using windows10.
Thank you!
It's easy you should write in your prompt template = """Answer the question based ONLY on the following context:
{context}
Question: {question}
"""
@@ibrahimgamal7603 Could be anything such as you are not using a good doc extractor or your embedding model sucks. Changing chunk and overlap size might help. Don't go for finetuning straight away, do some experimentation and figure out where the problem lies exactly otherwise even finetuning won't work.
@@muhammadumarnawaz9200 ok mate thank you for your help
wouldve been nice if you had shared the full collab code...
Guys, it is a collab from Unsloth. You can search on Google "Llama-3.1 8b unsloth" and you will find out.
Hey, I have a question. I'm currently using gpt api tokens to generate models response thru python and run it over and over. Models are using information from knowledge base folder -- it would be easier to manage it through llama? My ideal scenario is to generate content model, by model and then use that output to generate complete story. Which solutions will you recommend for that purpose? Thanks
Great video! Subscribed! 🎉
Another great video!! Thanks.
So we are just creating loras here, I can have many and mount them as I need like with Stable Diffusion right?
Intresting video!!
Well explained. Thanks
Thank you for video. I followed your approach for finetuning the model text to SQL. When I work on my own database, its performance is not good. Unable to generate SQL query. I even used agent and examples queries our database. I am requesting suggestion from you. Thank you
Awesome explanation in short video….some RUclipsrs consuming 2 to 3 hours for this content
Does this support AdaLoRA or Vera?
good guide to play by myself
I would appreciate if this tutorial went more into detail on how to make a dataset comply with the model's promt template. From this video, there is no data processing being done and i think in real cases you would have to work with the data, or write a script, that chnges the data so that it fits with the prompt template (e.g. the Alpaca one she shows in the video)
is the notebook posted anywhere?
How do you train on 3Gb pdf and word files locally
Is unsloath free?
if I train a model, will the responses be restricted to just my training data/examples? Or does the model's original training also play a role?
For example, say I trained it on model to respond with vehicle specs. I feed it a vehicle name and the model returns the vehicle specs. Say I finetune the model with 50 examples (honda civic, toyota camry, volvo xc90, etc).... however, in reality there are 300,000 possible vehicles. Would finetuning on 50 examples help? Sure, I can put the 50 models in the prompt but then I'd just be wasting tokens.
Can you provide me the link for your ubuntu desktop wallpaper 🙃
Can you train 12B on 24GB or is 12B too big?
Another question is if you have multi-turn data (conversations) can you finetune on that? The examples I see are for Q:A pairs.
Wow! Can this work on MAC?
I actually liked the music. Great editing also. I found the proyect idea a little bit boring. You could train it for something actually awesome
Yoo, what theme are you using in your system?? That's really cool
Isn't this RAG? Doesn't chunking fit anywhere?
can this be done all from the wsl terminal? I am using ollama, webui and docker, but would like to train some company pdf's better, can this be done?
what is the estimate cost for training such a model?
I have 55K classes with an ungodly amount of transcripts. What'll be better and faster? RAG or finetuning?
Apologies if this is a dumb question, but where is, is there a link to the notebook?
Great tutorial! Thanks. Really like the definitions and explanations rather than just glossing over the code.. would love to see a full tutorial series 😮
I'd like to try to code it myself, but it would be good if you could share the code too.
Do you have github repo of the code?
Does fine-tuning shouldn't generate small models that theoretically run fully local and avoid spending money on OpenAI?
It was a great video, but I have a question, is it compatible with the new versions called 3.2, especially versions 1B and 3B?
Excellent tutorial! Doesn’t lowering the bit depth of the model greatly reduce accuracy? What are the pros and cons of doing so? Thanks!
Where is the data? I did not see any data given to the model to use. I feel like its perfectly possible to have a model "train in realt time" where you tell a model go in to learning mode. Then it will "learn" or "memorize" data you give it. Specifically text based data. And then it has a chance of using it when you ask a question related to that data. Why has this not been done yet? Or has it? I guess what im talking about is somthing like custom GPTs. Which i feel would be eaiser and more worth it. Its just im looking for that but on device.
Is it possible to make your own AI model with this that speaks to you like Jarvis and understands tone, etc?
fine tuneing offline ?
what ubuntu are you using, the terminal looks dope
that subselect that it generated @5:01 makes me instinctively cringe - hoping it would generate the subselect as a join... - for instance why do:
"select p.* from posts as p inner join posttags as pt on p.post_id = pt.post_id where tag_id in ( select tag_id from tags where name = 'terminal' )"
instead of:
select p.* from posts p inner join posttags pt on p.post_i = pt.post_id inner join tags t on t.tag_id = pt.tag_id where t.name = 'terminal'
seriously - the 2nd one appears easier to optimize. right? oh well....
How long did it take for the training / fine-tuning on your 4090? Thanks for the video!
Thanks 🙏
How to collect dataset pls show it
Excellent direct video on fine-tuning, congrats. Could you also share the python source code that you used in the video? Thanks.
Nice video, but the music is way too disctracting.
nah, only a little distracting.
Video on formatting data sets to follow? 😅
Can we train tinyllama to do something similar? Since was trying to run AI on Raspberry Pi 5 (with Hailo AI Accelerator)
Absolutely!
@@warpdotdev Thanks! ❤️ from India
subscribed
Can you make the code are smaller and remove the background music
It's a great how to but you missed a "before trainng" and "after training" examples of how the model responded. Good video anyway. Couldn't get much out of it though
I think I'll just get an llm to make the dataset and make it llms both sides of the data->training loop
Here it is, she says.. For simplicity.... A pip install command.. In a video.. With no pasted text to copy in the description.
That's the opposite of simplicity.
Inam finding difficulty understanding the converting of data set
Is ubuntu on windows ? Or dependent system ?
I need more tiny steps to convert the dataset
Unsloth has a dependency of triton, which doesn't seem to be compatible with windows.
damn, now i want to create my own personal DAN assistant without OpenAI interfere with her
Thanks! Although I do have a question why you show so much of yourself... I admit it's nice and pleasant to look at, however I would prefer to look at the code or things you talk about. Especially when I watch your videos at work I would definitely like to avoid someone thinking that I watch some silly videos during work while I actually watch work related stuff. Anyway, really interesting videos so thanks again! 😊✌
Give us the collab link! Or better yet the llamasql gguf link!
why didn't you directly use hugginface trainer to train, why using unsloth? I want to know what was the benefit of using unsloth over hugginface trainer.
No benefit My friend :(
Less memory was what she stated
Nice
Can I do this without Conda? I hate Conda
Why yes, why wouldn’t you?
She's cute and I can now fine tune my llama 3. Yay 🎉
link
We need a video to explain this video
Why not share the source code???
Congrats for finding such a smartie-cutie as a DevRel for Warp
🙏
You move and changing Screen very fast we understand nothing and Result and where file store and after that you merge with Ollama model nothing understand.
cant use claude!!!!
Thankyou sis
Why are there so many videos that are useless as tutorials and give the impression that they are only about self-promotion? Seriously, anyone who understands this quick run-through here doesn't need any more tutorials, and for the vast majority of the rest, it's probably pretty useless.
let us use ollama with warp ai :(
Rodriguez John Thomas Lisa Rodriguez Jose
Good video, but it’s hard to imagine that you you actually sat there and edited out every breath and pause so that the whole video would sound like one massive run on sentence, only to shave off some 20 or 30 seconds of duration.
Gonzalez Susan Walker Larry Martinez Joseph
where is the link for google colab?