You have the best AI content out there hands down. While other people are just out there parroting the obvious you are actually teaching how this stuff really works. I thank you!
Just 15 minutes in, and you have taught me things I didn't know, and taught them in such a way that I understood every single point as if it were obvious. Love this video!
This is very cool, and indeed similar to my recent approach to data generation for gpt3.5 fine tuning (needed milti-lingual support). It would be great to evaluate the diff in performance between FT and RAGs, as in theiry RAGs might be easier to manage when data changes more often, right?
Yes, definitely RAG generally the preferred option. I'm just separating here and focusing on fine-tuning, will be back fairly soon with more on the RAG front.
It will also be subject to hallucination, but you have two options: 1. Train and prompt the llm to only answer based on info provided by the context/RAG 2. add some rows of data with "wrong/difficult/unknown" questions and have the answer be "I don't know". Probably less robust
@@TrelisResearch I'm finetuning the model to memorize certain data, so I should probably go with the second approach. Thanks. Your videos are immensely helpful!
Wow, that's such an excellent tutorial. But I have a question, is the process of generating testing samples the same as training samples? Is the difference between the two just a difference in the number of question and answer pairs generated?
I'm not intimately familiar with MemGPT but it involves taking your conversation history and storing it in a database so it can be reused. By comparison, if you fine-tuning, you are actually updating the model weights to incorporate the info (which will be faster as an approach, but takes the effort and skill to actually do the fine-tuning).
Hello, I was manually putting chat_template for Llama3 myself before I watched this video. I try messages format now storeing all messages as a string in huggingface dataset but my model dosen't learn well that way. It outputs system roles Instruction sometimes and mostly it converse with it self it thinks all roles is its role. How can I fix this if you can you recommend something I will be appriciate it thx for the video btw.
howdy, yeah I'd recommend starting off with a really simple example with just a few rows of data and trying that on a small model. you can also check out the livestream videos on this channel, there is one where I cover chat templates
In all the example implementations, may it be with embeddings or fine tuning, QA is usually posed to use cases with factual data like rule book or manuals or stuff like that. What about very complex and multilayered texts like history or philosophy? For example if we use GPT4 to chat with a book like History of Western Philosophy, it will reply that who is the author and other factual questions but won't be able to carry out a deep discussion keeping material of the book in context. So can we use LLMs as memory machines of Philosophy or History tomes?
Can this same strategy of synthetic data be effective for fine-tuning gpt on openai platform? So basically consider this: I have a short story(around 10k tokens long). I have divided the entire story into list of situations and incidents(around 13). For each incidents I have the original text from the story. Now I create questions from different angles for each incidents. When a user asks query to my system, will the fine-tuned gpt be able to identify the incident in which this question lies? What I think is that fine-tuning and iverfitting on rugby rules is quite easier that on some story or novel. Becasue in case of rugby rules you have one word or one-liner answer. In case of stories their is context in the first paragraph as well as the ending paragraphs too and therefore it makes it more tough imo. For e.g. If you had to ask about the nature of Harry Porter, there is no one line answer, the model needs to know context from various paragraphs before answering it Correct me if I am wrong and if possible can you try memorization of novel or a story, I think its a more intelligent system that QnA on set of rules. Thanks!
Yes, I think the same concept can be used, and I agree you are right that it will take more (and longer) synthetic data to achieve the same effect. Starting with situations/incidents/plot-points makes good sense.
@@adidevbhattacharya9220 it's a good thought, but probably be a while before I come back around to the storytelling use case, as I've laid out the basic approach here and people can build on that. Probably it makes sense to do unsupervised fine-tuning on the novel and then additionally do this supervised fine-tuning approach with some plot points from each chapter.
a) The LLM needs to see the knowledge from different angles (otherwise it will only be fine-tuned on those exact phrases and won't generalise as well b) if you just fine-tune on the pdf, the LLM will start losing the ability to respond in chat format (because the pdf is not chat format).
You have the best AI content out there hands down. While other people are just out there parroting the obvious you are actually teaching how this stuff really works. I thank you!
big thanks
Just 15 minutes in, and you have taught me things I didn't know, and taught them in such a way that I understood every single point as if it were obvious. Love this video!
Cheers
Awesome, always a good day when you upload. Thanks again good sir!
This guy is a professor? excellent!!!!
Awesome video and very good learning material both for beginners and more advanced! Thank you!
New to fine tuning and found this extremely helpful. Thank you for putting it together.
You're welcome, thanks!
You create very high quality content. Thank you so much.
you're welcome
awesome, was waiting for this
Wow I’d appreciated rephrasing for data augmentation in general but not specifically to handle the reversal curse. Brilliant insight!
this is really useful, the best thing i found on youtube
thanks, appreciate it
amazing video guide. thank you man!
you're welcome
Wow! Glad I found you
This is very cool, and indeed similar to my recent approach to data generation for gpt3.5 fine tuning (needed milti-lingual support). It would be great to evaluate the diff in performance between FT and RAGs, as in theiry RAGs might be easier to manage when data changes more often, right?
Yes, definitely RAG generally the preferred option. I'm just separating here and focusing on fine-tuning, will be back fairly soon with more on the RAG front.
Awesome. Congrats 🎉
thanks
Awesome tutorials, we should be friends😀
Excellent
Is there a way to finetune LLM to say I don't know? This is very important for domain-specific LLMs. Thanks.
It will also be subject to hallucination, but you have two options:
1. Train and prompt the llm to only answer based on info provided by the context/RAG
2. add some rows of data with "wrong/difficult/unknown" questions and have the answer be "I don't know". Probably less robust
@@TrelisResearch I'm finetuning the model to memorize certain data, so I should probably go with the second approach. Thanks. Your videos are immensely helpful!
Awasome video
Wow, that's such an excellent tutorial. But I have a question, is the process of generating testing samples the same as training samples? Is the difference between the two just a difference in the number of question and answer pairs generated?
Yes! And yea you want more samples for training. For eval you want enough to be statistically significant when measuring improvement
@@TrelisResearch Thanks!!! It really helped me a lot!!! But how do you ensure the quality of generated dataset? Is there a method to evaluate?
@@嗟怀人 Well if your training run scores well on evaluation / validation questions, then you know it's good!
Thanks for this tutorial! How does this approach compare to something like MemGPT?
I'm not intimately familiar with MemGPT but it involves taking your conversation history and storing it in a database so it can be reused. By comparison, if you fine-tuning, you are actually updating the model weights to incorporate the info (which will be faster as an approach, but takes the effort and skill to actually do the fine-tuning).
Thank you!@@TrelisResearch
As always, great content! But why is the video quality limited to 720p?
hmm, thanks, I'm going to check that going forward and aim to upload higher.
ok, yeah, thanks, I realised my computer camera is 720p, I'm going to shoot with a better camera from now on.
@@TrelisResearch The camera quality might be 720p but the content quality is 4k!
Hello, I was manually putting chat_template for Llama3 myself before I watched this video. I try messages format now storeing all messages as a string in huggingface dataset but my model dosen't learn well that way. It outputs system roles Instruction sometimes and mostly it converse with it self it thinks all roles is its role. How can I fix this if you can you recommend something I will be appriciate it thx for the video btw.
howdy, yeah I'd recommend starting off with a really simple example with just a few rows of data and trying that on a small model. you can also check out the livestream videos on this channel, there is one where I cover chat templates
Can I use this fine-tuning script for Llam2 models?
Yes, definitely
Great stuff, as always.
How good would this approach be for text inviting higher order QA such as philosophy or sociology texts?
Thanks.
Hmm, could you reframe your question, maybe with an example - to help me better respond
In all the example implementations, may it be with embeddings or fine tuning, QA is usually posed to use cases with factual data like rule book or manuals or stuff like that.
What about very complex and multilayered texts like history or philosophy? For example if we use GPT4 to chat with a book like History of Western Philosophy, it will reply that who is the author and other factual questions but won't be able to carry out a deep discussion keeping material of the book in context.
So can we use LLMs as memory machines of Philosophy or History tomes?
Can this same strategy of synthetic data be effective for fine-tuning gpt on openai platform?
So basically consider this:
I have a short story(around 10k tokens long). I have divided the entire story into list of situations and incidents(around 13). For each incidents I have the original text from the story. Now I create questions from different angles for each incidents.
When a user asks query to my system, will the fine-tuned gpt be able to identify the incident in which this question lies?
What I think is that fine-tuning and iverfitting on rugby rules is quite easier that on some story or novel. Becasue in case of rugby rules you have one word or one-liner answer. In case of stories their is context in the first paragraph as well as the ending paragraphs too and therefore it makes it more tough imo.
For e.g. If you had to ask about the nature of Harry Porter, there is no one line answer, the model needs to know context from various paragraphs before answering it
Correct me if I am wrong and if possible can you try memorization of novel or a story, I think its a more intelligent system that QnA on set of rules.
Thanks!
Yes, I think the same concept can be used, and I agree you are right that it will take more (and longer) synthetic data to achieve the same effect. Starting with situations/incidents/plot-points makes good sense.
Can you maybe prepare a brief guide on the steps(or a video) for giving knowledge base of some novel or story.
Thanks@@TrelisResearch
@@adidevbhattacharya9220 it's a good thought, but probably be a while before I come back around to the storytelling use case, as I've laid out the basic approach here and people can build on that.
Probably it makes sense to do unsupervised fine-tuning on the novel and then additionally do this supervised fine-tuning approach with some plot points from each chapter.
Hy. What about teaching the model some definitions that must be given to the user as it is without changing the answer or generalised.
hmm, can you clarify your question here?
a noob question: why are you not training on the PDF itself but converting to chat and training on the chat?
a) The LLM needs to see the knowledge from different angles (otherwise it will only be fine-tuned on those exact phrases and won't generalise as well
b) if you just fine-tune on the pdf, the LLM will start losing the ability to respond in chat format (because the pdf is not chat format).
Another great video thanks: (BhamDeFiGuy)
Cheers
batch size 0.o