Your code is very helpful. I have the following questions 1. What should the format of the document be? Should it be question answer format or just paragraphs and let me model learn the text? 2. If we want the chatbot to take questions and provide answers, how should the data be formatted to ensure only the answer is provided and not the entire text? I am facing an issue where I am getting text beyond the expected answer.
How similar to the source material (structure wise?) are the responses from the medium and larger model? From experimenting I've found that the GPT-2s don't do much "thinking" and are extremely dependent on your prompt lining up with the training materials..
Hello everything is fine? your videos are great, very explanatory, but I would like you to post the format of the texts you used to do the fine tuning, as I am having difficulties receiving cohesive responses with my fine tuned model, if possible
Thanks for the video! I get this error though. Ive installed the dependencies but the error doesnt go away. Has anyone experienced the same? ImportError: Using the `Trainer` with `PyTorch` requires `accelerate>=0.20.1`: Please run `pip install transformers[torch]` or `pip install accelerate -U`
Hello sir I would like to ask you the following. If my understanding is correct you are basically fine tune the gpt LM on your specific context. In order to ask question relative to the context we provide in the pdfs shouldn't we trained (fine-tune ) the model on the question answering task using for example the Squad dataset? If this is the case how can we do it? Also how can we do it for other languages other than English? My native language is Greek.
Thanks sir, i want to modify the gpt-2 model for the supervising task, using prompts, input as question and output is answer, is it possible. Do you have any recommendations,
Yes. That is what chatGPT does. Also, one textbook is not enough, you need to train it on almost the entire GitHub for it to be able to write usable code right out of the box.
Hi, Can we train GPT-3 model with a 100-page law book to get answers using 16GB GPU? Another question is how can we limit the answers specific to the books which we trained to avoid general questions and answers?
with open("train.txt", "w",encoding="utf-8") as f: f.write(train_text) with open("val.txt", "w", encoding="utf-8") as f: f.write(val_text) replace thos two line hopefully it will work just use encoding="utf-8
Let’s say I used chat engine to help generate the code, why is that sad? I recommend using any help you can get in putting your code together. Do not waste time in writing code from scratch. Focus on solving a problem, not rewriting code. Chat bots can easily generate code that’ll get you 80% there, you need to finish it off by customizing for your specific task. Never be ashamed of using Chabot, would you be ashamed of using a calculator or a search engine (e.g., Google) to solve a challenge?
Thank you Sreeni! Really appreciate your lessons!
Thank you very much Will, I really appreciate the generosity.
I am really curious to better understand how tokenization happens. I'd love to watch any video you make on that topic.
Tokenization video next Wednesday :)
hey i am having some error while passing the path of directory if u implemented the code then can you please assist me in this error
Your code is very helpful. I have the following questions
1. What should the format of the document be? Should it be question answer format or just paragraphs and let me model learn the text?
2. If we want the chatbot to take questions and provide answers, how should the data be formatted to ensure only the answer is provided and not the entire text? I am facing an issue where I am getting text beyond the expected answer.
How similar to the source material (structure wise?) are the responses from the medium and larger model? From experimenting I've found that the GPT-2s don't do much "thinking" and are extremely dependent on your prompt lining up with the training materials..
Hello everything is fine? your videos are great, very explanatory, but I would like you to post the format of the texts you used to do the fine tuning, as I am having difficulties receiving cohesive responses with my fine tuned model, if possible
Thank you Sreeni! so great lessons.
Thank you for sharing. However, I have a question how to enhance the model to avoid the issue with separating the words.
Where do you define the eos_token and bos_token in the training text formatting? Also, what is the format of the dataset txt files?
Thanks !
I cannot find the code in the GitHub (309 - Training your own Chatbot using GPT)
github.com/bnsreenu/python_for_microscopists/blob/master/309_Training_your_own_Chatbot_using_GPT%E2%80%8B.ipynb
@@DigitalSreeni Thanks!
hey i am having some error while passing the path of directory if u implemented the code then can you please assist me in this error
Thank you! This information has been essential to me for a project.
Great video once again - some pretty scary stuff going on at 20:39 lol
Just trying to scare you :)
Thanks for ur videos, sir
hey i am having some error while passing the path of directory if u implemented the code then can you please assist me in this error
how to prevent generate_response result not complete sentence and repeated senetence?
Thanks for the video! I get this error though. Ive installed the dependencies but the error doesnt go away. Has anyone experienced the same? ImportError: Using the `Trainer` with `PyTorch` requires `accelerate>=0.20.1`: Please run `pip install transformers[torch]` or `pip install accelerate -U`
NVM I just needed to restart the runtime 🙈
OMG .... THAT REALLY AMAZING.. THANKS YOU SO MUCH!!
Thank you for this video! Would you please consider making a video about the implementation of LLaMA (architecture) or any other LLM?
hey i am having some error while passing the path of directory if u implemented the code then can you please assist me in this error
@@ananayaggarwal7909 which code !?
Hello sir I would like to ask you the following. If my understanding is correct you are basically fine tune the gpt LM on your specific context. In order to ask question relative to the context we provide in the pdfs shouldn't we trained (fine-tune ) the model on the question answering task using for example the Squad dataset? If this is the case how can we do it? Also how can we do it for other languages other than English? My native language is Greek.
hey i am having some error while passing the path of directory if u implemented the code then can you please assist me in this error
Thanks sir, i want to modify the gpt-2 model for the supervising task, using prompts, input as question and output is answer, is it possible. Do you have any recommendations,
Can I use this neural network (GPT-2) to train with Spanish text and get answers in Spanish (Spanish output)?
Hi sir
I am planning to develop a healthcare chatbot so where can I get data and which SOTA Model is best to give good results using NLP
Can I train it by providing it a programming textbook then ask it to write me some code based on what it learned from the book?
🤔
Yes. That is what chatGPT does. Also, one textbook is not enough, you need to train it on almost the entire GitHub for it to be able to write usable code right out of the box.
hey i am having some error while passing the path of directory if u implemented the code then can you please assist me in this error
Hi, Can we train GPT-3 model with a 100-page law book to get answers using 16GB GPU? Another question is how can we limit the answers specific to the books which we trained to avoid general questions and answers?
hey i am having some error while passing the path of directory if u implemented the code then can you please assist me in this error
you dont get the gpt3 weights, only OpenAI has them :)
Are WizardLM, Alpaca, vacuna, etc. Better than all versions of gpt2 why not to use these ones?
hey i am having some error while passing the path of directory if u implemented the code then can you please assist me in this error
Can you please include code for reading exel files in the directory. Thanks in advance
How many files did you use for training???
Excellent!!
hey i am having some error while passing the path of directory if u implemented the code then can you please assist me in this error
@@ananayaggarwal7909 You must have your own folder with files to pass to the directory. He has just shown a symbolic path for his own folder.
Hey, can't find the PDF
Thx
hey i am having some error while passing the path of directory if u implemented the code then can you please assist me in this error
@@ananayaggarwal7909 did your error solved i am having similar error??
Sir can it train tif jp2 image in chargpt chatGPt.? Thanks!
ChatGPT is a language model, so you cannot train it on images.
hey i am having some error while passing the path of directory if u implemented the code then can you please assist me in this error
Can you please provide the dataset link of it
hey i am having some error while passing the path of directory if u implemented the code then can you please assist me in this error
@@ananayaggarwal7909 i am having the same error
sir it showing directory error when i input path of txt file
with open("train.txt", "w",encoding="utf-8") as f:
f.write(train_text)
with open("val.txt", "w", encoding="utf-8") as f:
f.write(val_text)
replace thos two line hopefully it will work
just use encoding="utf-8
Bro used ChatGPT to generate the python script he's showing how sad is that
Let’s say I used chat engine to help generate the code, why is that sad? I recommend using any help you can get in putting your code together. Do not waste time in writing code from scratch. Focus on solving a problem, not rewriting code. Chat bots can easily generate code that’ll get you 80% there, you need to finish it off by customizing for your specific task. Never be ashamed of using Chabot, would you be ashamed of using a calculator or a search engine (e.g., Google) to solve a challenge?