The more I'm seeing of A.I. advancement, I'm coming to the concluding that better isn't always better, and the real battleground isn't so much which is the best when many of the better A.I. models are so close to each other in quality, the real battleground for me is the quality for the size, so a bit like we do for hardware, performance per watt, but in this case, performance per billion parameters, if you can maintain or have better quality at a smaller size, that is a major advantage, especially if it's open source and can run locally on your hardware. So as good as the big A.I. models are, they are too tightly controlled and very limited in how you can run them, in most cases online because of how big they are, the real game changer I think is with the smaller open source models that you can run locally, the advantage they've got is that they can be fully integrated and specialised in the OS, apps and games, they also have the advantage of less privacy, security and other concerns like that. If the current advancements of A.I. models continues and hardware continues to progress, I suspect the online big models are not going to matter that much as the smaller ones we can run locally will be able to do most of the things we want, and that's when things get really interesting as A.I. gets far more integrated into our daily lives, something that's really limited with these online centralised A.I. models and for countless reasons. At the end of the day, what's going to win out isn't going to be the best, good enough will do for most of us, what will really win out is what is smaller, capable and can be run locally, which basically rules out the big online A.I. services as there are too many privacy and security concerns with them, especially as A.I. becomes more capable and integrated into our lives.
Thank you so much for this wonderful video! I have a couple of questions: For max_seq_length = 2048 # Choose any! We auto-support RoPE scaling internally!, could you clarify, whether it handles cases where the input sequence length exceeds 2048 tokens? Also, when determining the max sequence length for custom data, should it include the combined length of the instruction, input, and output? Thank you again for your insights!
hello sir !!!! wonderful contribution!!! can you practically train the model on the data so that we can learn . I am new to this field and your channel is amazing. thanks
This is so funny, been looking for this yesterday and today now. Maybe I'm just now realizing after 20 years of google searching experience that I'm bad at googling.
Nice . Can we run this in our local machine and what config needed to run in local mackbook. Or colab is preferred please let me know.Also can you suggest is mackbook good for handling LLMS
Training should be conducted on a CUDA device, but the resulting model can be used on MPS devices (MacBook M series) and CPUs. For fine-tuning models on Mac using MLX-a powerful, open-source array framework for Apple silicon-there's a vibrant community supporting it.
Could you make a video on how to create a training set to fine-tune a model? I want to fine-tune a model like LLAMA-3.1 that creates YAML sections for different tasks similar to ansible. For example when I prompt: "Create a user alice" it should generate a YAML in a specific format like user: action: create username: alice Can you show how we can create such a training set. I can't create thousands of training data manually.
@@engineerprompt Could you give an example? You mean like explain the format of the YAML file, make an example and e.g. write "Whenever I create an user, output this YAML file"?
Is it normal that the fined tuned version response with the ### Instruction, ### Input, ### Response pattern. Do I have a alternative in the training section, when i want only the response?
nice video but as most of the other in the same topic use an all ready dataset... i would prefer to see a video juat for a basic construction of a custom alpaca dataset... I think is what is missing from the most of the same kind tutorials.. the logic and the method to create your own alpaca dataset, what if a question has more than one answer? what if a simple question need to be clarified by the user depending of two probabilities ? and then follows the answer based on the clarification user inputs etc ....
When I am using this code "model.push_to_hub_merged("My_Modal_Path", tokenizer, save_method="merged_16bit")" it shows this error "TypeError: argument of type 'NoneType' is not iterable". All files are saved successfully, but when unsloth trying to upload it shows this error.
The most complex problm is preparing dataset with QnA from which u gonna learn. And this is what I'd like to see.
here is a previous video I did on creating custom datasets: ruclips.net/video/z2QE12p3kMM/видео.html
Fascinating! The two Australian brothers did a fantastic job of introducing the Unsloth to the community.
Agree, they are doing great job.
The more I'm seeing of A.I. advancement, I'm coming to the concluding that better isn't always better, and the real battleground isn't so much which is the best when many of the better A.I. models are so close to each other in quality, the real battleground for me is the quality for the size, so a bit like we do for hardware, performance per watt, but in this case, performance per billion parameters, if you can maintain or have better quality at a smaller size, that is a major advantage, especially if it's open source and can run locally on your hardware.
So as good as the big A.I. models are, they are too tightly controlled and very limited in how you can run them, in most cases online because of how big they are, the real game changer I think is with the smaller open source models that you can run locally, the advantage they've got is that they can be fully integrated and specialised in the OS, apps and games, they also have the advantage of less privacy, security and other concerns like that.
If the current advancements of A.I. models continues and hardware continues to progress, I suspect the online big models are not going to matter that much as the smaller ones we can run locally will be able to do most of the things we want, and that's when things get really interesting as A.I. gets far more integrated into our daily lives, something that's really limited with these online centralised A.I. models and for countless reasons.
At the end of the day, what's going to win out isn't going to be the best, good enough will do for most of us, what will really win out is what is smaller, capable and can be run locally, which basically rules out the big online A.I. services as there are too many privacy and security concerns with them, especially as A.I. becomes more capable and integrated into our lives.
You have no idea how this video helped me!! THANK YOU SO MUCH
Thank you so much for this wonderful video! I have a couple of questions: For max_seq_length = 2048 # Choose any! We auto-support RoPE scaling internally!, could you clarify, whether it handles cases where the input sequence length exceeds 2048 tokens? Also, when determining the max sequence length for custom data, should it include the combined length of the instruction, input, and output? Thank you again for your insights!
Excellent explanation!!
Thanks for video, every minute detailed video .superb.
Thank You Brother, Truly
Haven't watched yet but thank you for all your guides on this, I know where to come when I need to do this myself !!
thank you!!!
like this and this show the easy way for ppls who not are student for ai . not newbie frendly to complex tutorial
This is awesome, and tutorial is so easy to understand too
:)
Thank you 😀
hello sir !!!! wonderful contribution!!! can you practically train the model on the data so that we can learn . I am new to this field and your channel is amazing. thanks
here is a previous video I did on creating custom datasets: ruclips.net/video/z2QE12p3kMM/видео.html
Thank you for this video tutorial very helpful!
I have problem in creating my own datasets manually. Like CSV file format, how can structure it in CSV file and read it to the fine tunning process?
Thanks for video!!
Can you inform me how to deploy a fine tuned model?
check out this playlist on deployment:
ruclips.net/video/OuQBxBrO2ms/видео.html&ab_channel=PromptEngineering
why nobody speaks about, "How should we convert my CONFIDENTIAL RAW text/ PDF into Datasets"????
This is so funny, been looking for this yesterday and today now. Maybe I'm just now realizing after 20 years of google searching experience that I'm bad at googling.
here is a previous video I did on creating custom datasets: ruclips.net/video/z2QE12p3kMM/видео.html
Thank you!!
Nice . Can we run this in our local machine and what config needed to run in local mackbook. Or colab is preferred please let me know.Also can you suggest is mackbook good for handling LLMS
Training should be conducted on a CUDA device, but the resulting model can be used on MPS devices (MacBook M series) and CPUs. For fine-tuning models on Mac using MLX-a powerful, open-source array framework for Apple silicon-there's a vibrant community supporting it.
Could you make a video on how to create a training set to fine-tune a model? I want to fine-tune a model like LLAMA-3.1 that creates YAML sections for different tasks similar to ansible. For example when I prompt: "Create a user alice" it should generate a YAML in a specific format like
user:
action: create
username: alice
Can you show how we can create such a training set. I can't create thousands of training data manually.
You can use function calling also it may be enough rather than fully finetuneing
You can achieve this through prompting. fine-tuning should be a last resort. You dont' need it in most cases.
@@engineerprompt Could you give an example? You mean like explain the format of the YAML file, make an example and e.g. write "Whenever I create an user, output this YAML file"?
hey can you please explain how to fine to model and deploy to own server if privacy is concern. please make a tutorial on it
Here is a playlist on deployments:
ruclips.net/video/OuQBxBrO2ms/видео.html&ab_channel=PromptEngineering
Thanks for sharing
Hello bro how r u. I just started here but confused where to begin. Can you guide me in a specific direction.
Thank you :)
what exactly is your confusion. Are you interested in getting started with LLMs or fine-tuning them?
@@engineerprompt which should I start 1st and from which video or playlist should I start?
Are we able to fine tune the model directly which is available in the ollama server?
Is it normal that the fined tuned version response with the ### Instruction, ### Input, ### Response pattern. Do I have a alternative in the training section, when i want only the response?
The colab link doesn't seems to work
can i have a validation set in sfttrainer?
nice video but as most of the other in the same topic use an all ready dataset... i would prefer to see a video juat for a basic construction of a custom alpaca dataset... I think is what is missing from the most of the same kind tutorials.. the logic and the method to create your own alpaca dataset, what if a question has more than one answer? what if a simple question need to be clarified by the user depending of two probabilities ? and then follows the answer based on the clarification user inputs etc ....
here is a previous video I did on the topic: ruclips.net/video/z2QE12p3kMM/видео.html
@@engineerprompt Thanks for your reply.. just checked the Link, awesome video... Thanks for sharing your knowledge!!
the issues is that benchmark are broken, seeing the graph is pointless at this point
I agree but unfortunately that's the only thing we have at the moment.
@@engineerprompt well it is not true, you can craft your own benchmark, some people are doing it, and share with us what you thing from it
Can you show how can ı finetune it and store the new one locally. Like ı already have llama3.1 on my local. I want to finetune it and use it
Towards the end of the video, I show how to save and load the models locally. You can use that part of the code
Does unsloth support CPU configuration?
it needs GPU
@@engineerprompt Are we able to train the model which is available in ollama directly by taking that as base model?
You are not fine tuning a damn thing bro
Meaning… he wants a step by step vs a high level how to.
When I am using this code "model.push_to_hub_merged("My_Modal_Path", tokenizer, save_method="merged_16bit")" it shows this error "TypeError: argument of type 'NoneType' is not iterable". All files are saved successfully, but when unsloth trying to upload it shows this error.