You have the best AI content out there hands down. While other people are just out there parroting the obvious you are actually teaching how this stuff really works. I thank you!
This is very cool, and indeed similar to my recent approach to data generation for gpt3.5 fine tuning (needed milti-lingual support). It would be great to evaluate the diff in performance between FT and RAGs, as in theiry RAGs might be easier to manage when data changes more often, right?
Yes, definitely RAG generally the preferred option. I'm just separating here and focusing on fine-tuning, will be back fairly soon with more on the RAG front.
Hello, I was manually putting chat_template for Llama3 myself before I watched this video. I try messages format now storeing all messages as a string in huggingface dataset but my model dosen't learn well that way. It outputs system roles Instruction sometimes and mostly it converse with it self it thinks all roles is its role. How can I fix this if you can you recommend something I will be appriciate it thx for the video btw.
howdy, yeah I'd recommend starting off with a really simple example with just a few rows of data and trying that on a small model. you can also check out the livestream videos on this channel, there is one where I cover chat templates
Can this same strategy of synthetic data be effective for fine-tuning gpt on openai platform? So basically consider this: I have a short story(around 10k tokens long). I have divided the entire story into list of situations and incidents(around 13). For each incidents I have the original text from the story. Now I create questions from different angles for each incidents. When a user asks query to my system, will the fine-tuned gpt be able to identify the incident in which this question lies? What I think is that fine-tuning and iverfitting on rugby rules is quite easier that on some story or novel. Becasue in case of rugby rules you have one word or one-liner answer. In case of stories their is context in the first paragraph as well as the ending paragraphs too and therefore it makes it more tough imo. For e.g. If you had to ask about the nature of Harry Porter, there is no one line answer, the model needs to know context from various paragraphs before answering it Correct me if I am wrong and if possible can you try memorization of novel or a story, I think its a more intelligent system that QnA on set of rules. Thanks!
Yes, I think the same concept can be used, and I agree you are right that it will take more (and longer) synthetic data to achieve the same effect. Starting with situations/incidents/plot-points makes good sense.
@@adidevbhattacharya9220 it's a good thought, but probably be a while before I come back around to the storytelling use case, as I've laid out the basic approach here and people can build on that. Probably it makes sense to do unsupervised fine-tuning on the novel and then additionally do this supervised fine-tuning approach with some plot points from each chapter.
In all the example implementations, may it be with embeddings or fine tuning, QA is usually posed to use cases with factual data like rule book or manuals or stuff like that. What about very complex and multilayered texts like history or philosophy? For example if we use GPT4 to chat with a book like History of Western Philosophy, it will reply that who is the author and other factual questions but won't be able to carry out a deep discussion keeping material of the book in context. So can we use LLMs as memory machines of Philosophy or History tomes?
I'm not intimately familiar with MemGPT but it involves taking your conversation history and storing it in a database so it can be reused. By comparison, if you fine-tuning, you are actually updating the model weights to incorporate the info (which will be faster as an approach, but takes the effort and skill to actually do the fine-tuning).
a) The LLM needs to see the knowledge from different angles (otherwise it will only be fine-tuned on those exact phrases and won't generalise as well b) if you just fine-tune on the pdf, the LLM will start losing the ability to respond in chat format (because the pdf is not chat format).
You have the best AI content out there hands down. While other people are just out there parroting the obvious you are actually teaching how this stuff really works. I thank you!
big thanks
This guy is a professor? excellent!!!!
New to fine tuning and found this extremely helpful. Thank you for putting it together.
You're welcome, thanks!
Awesome, always a good day when you upload. Thanks again good sir!
Wow I’d appreciated rephrasing for data augmentation in general but not specifically to handle the reversal curse. Brilliant insight!
awesome, was waiting for this
amazing video guide. thank you man!
you're welcome
Excellent
Awesome. Congrats 🎉
thanks
This is very cool, and indeed similar to my recent approach to data generation for gpt3.5 fine tuning (needed milti-lingual support). It would be great to evaluate the diff in performance between FT and RAGs, as in theiry RAGs might be easier to manage when data changes more often, right?
Yes, definitely RAG generally the preferred option. I'm just separating here and focusing on fine-tuning, will be back fairly soon with more on the RAG front.
Wow! Glad I found you
Awasome video
Hello, I was manually putting chat_template for Llama3 myself before I watched this video. I try messages format now storeing all messages as a string in huggingface dataset but my model dosen't learn well that way. It outputs system roles Instruction sometimes and mostly it converse with it self it thinks all roles is its role. How can I fix this if you can you recommend something I will be appriciate it thx for the video btw.
howdy, yeah I'd recommend starting off with a really simple example with just a few rows of data and trying that on a small model. you can also check out the livestream videos on this channel, there is one where I cover chat templates
Hy. What about teaching the model some definitions that must be given to the user as it is without changing the answer or generalised.
hmm, can you clarify your question here?
As always, great content! But why is the video quality limited to 720p?
hmm, thanks, I'm going to check that going forward and aim to upload higher.
ok, yeah, thanks, I realised my computer camera is 720p, I'm going to shoot with a better camera from now on.
@@TrelisResearch The camera quality might be 720p but the content quality is 4k!
Can this same strategy of synthetic data be effective for fine-tuning gpt on openai platform?
So basically consider this:
I have a short story(around 10k tokens long). I have divided the entire story into list of situations and incidents(around 13). For each incidents I have the original text from the story. Now I create questions from different angles for each incidents.
When a user asks query to my system, will the fine-tuned gpt be able to identify the incident in which this question lies?
What I think is that fine-tuning and iverfitting on rugby rules is quite easier that on some story or novel. Becasue in case of rugby rules you have one word or one-liner answer. In case of stories their is context in the first paragraph as well as the ending paragraphs too and therefore it makes it more tough imo.
For e.g. If you had to ask about the nature of Harry Porter, there is no one line answer, the model needs to know context from various paragraphs before answering it
Correct me if I am wrong and if possible can you try memorization of novel or a story, I think its a more intelligent system that QnA on set of rules.
Thanks!
Yes, I think the same concept can be used, and I agree you are right that it will take more (and longer) synthetic data to achieve the same effect. Starting with situations/incidents/plot-points makes good sense.
Can you maybe prepare a brief guide on the steps(or a video) for giving knowledge base of some novel or story.
Thanks@@TrelisResearch
@@adidevbhattacharya9220 it's a good thought, but probably be a while before I come back around to the storytelling use case, as I've laid out the basic approach here and people can build on that.
Probably it makes sense to do unsupervised fine-tuning on the novel and then additionally do this supervised fine-tuning approach with some plot points from each chapter.
Great stuff, as always.
How good would this approach be for text inviting higher order QA such as philosophy or sociology texts?
Thanks.
Hmm, could you reframe your question, maybe with an example - to help me better respond
In all the example implementations, may it be with embeddings or fine tuning, QA is usually posed to use cases with factual data like rule book or manuals or stuff like that.
What about very complex and multilayered texts like history or philosophy? For example if we use GPT4 to chat with a book like History of Western Philosophy, it will reply that who is the author and other factual questions but won't be able to carry out a deep discussion keeping material of the book in context.
So can we use LLMs as memory machines of Philosophy or History tomes?
Thanks for this tutorial! How does this approach compare to something like MemGPT?
I'm not intimately familiar with MemGPT but it involves taking your conversation history and storing it in a database so it can be reused. By comparison, if you fine-tuning, you are actually updating the model weights to incorporate the info (which will be faster as an approach, but takes the effort and skill to actually do the fine-tuning).
Thank you!@@TrelisResearch
a noob question: why are you not training on the PDF itself but converting to chat and training on the chat?
a) The LLM needs to see the knowledge from different angles (otherwise it will only be fine-tuned on those exact phrases and won't generalise as well
b) if you just fine-tune on the pdf, the LLM will start losing the ability to respond in chat format (because the pdf is not chat format).
Can I use this fine-tuning script for Llam2 models?
Yes, definitely
batch size 0.o