Question Answer Generator App using Mistral LLM, Langchain, and FastAPI
HTML-код
- Опубликовано: 18 сен 2024
- Are you ready to dive into the world of cutting-edge natural language processing and open-source innovation? In this tutorial, I'll guide you through the creation of a powerful Question-Answer Generator App using Mistral 7B, a state-of-the-art Large Language Model, and the versatile Langchain framework, all running seamlessly on the CPU. 🚀
📚 What you'll learn:
How to harness the power of Mistral 7B, a revolutionary Large Language Model, for generating question-answer pairs from PDF documents.
Building a complete open-source application with Langchain as the foundation, perfect for those passionate about NLP and open-source development.
Practical applications:
Creating datasets for fine-tuning or training Large Language Models, generating quizzes, and enhancing your QA capabilities.
🔧 Key components used in this tutorial:
1. Sentence Transformers Model for efficient vector embeddings.
2. Faiss CPU for in-memory vector stores, ensuring blazing-fast performance.
3. FastAPI as the robust backend framework for seamless data processing.
Jinja2 for template rendering, making your app's user interface dynamic and user-friendly.
👨💻 Whether you're a seasoned developer or just starting your journey in the world of Generative AI, this tutorial will equip you with the knowledge and tools needed to create your own Question-Answer Generator App. Join me as we explore the limitless possibilities of Mistral 7B, Langchain, and open-source stack.
📢 Don't forget to like, share, and subscribe to stay updated with the latest in Gen AI and open-source development.
GitHub Repo: github.com/AIA...
LLM Playlist: • Large Language Models
QA Generator App using OpenAI: • Question and Answer Ge...
#generativeai #langchain #llm
Your video is literally 24 hours late for me. Same exact challenge was provided by college along with Summarizer and Content Chunking. I referred to your Chatbot video with local LLM and completed the project. Thanks for the content.
🎯 Key Takeaways for quick navigation:
00:00 🤖 Introduction to building a Question Answer generator app using open source tools
Key takeaways:
- The app will generate question and answer pairs from a given PDF document
- It will use Mistral, a new large language model, Langchain for orchestration, and FastAPI for the API
03:10 🛡️ Loading the model and preprocessing the PDF
Key takeaways:
- Loading the Mistral model with Transformers
- Preprocessing the PDF to extract text and split into chunks
08:38 💼 Creating the FastAPI endpoints
Key takeaways:
- Creating the index, upload, and analyze endpoints
- Upload endpoint saves the PDF and analyze generates the QA pairs
- Returning results as JSON using FastAPI response
13:47 🧠 Defining the LLMs and chains
Key takeaways:
- Creating the question generation and answer retrieval chains
- Using Langchain for orchestrating the LLM
- Storing embeddings in Fast vector store
23:32 📝 Writing the CSV output
Key takeaways:
- Writing a function to output the QA pairs to a CSV file
- Checking if output folder exists, creating it if needed
- Writing the rows with questions and answers
38:52 ▶️ Running the application
Key takeaways:
- Starting the FastAPI app server
- Uploading a PDF and generating QA pairs
-Downloading the CSV output file with questions and answers
Made with HARPA AI
Great video 👍 in the reverse use case u hava QA pairs stored in multiple documents. I know u can do fine tuningwith these pairs. But I have not seen yet a method to do RAG based on this QA documents....
Just what I needed! Thanks! Can you please make a video on how to train this model with Q/A pairs? Also with support on GPU?
Thanks a lot for the work
Thanks for the support.
If it's possible, please upload the code for the pet-care llm project. Definitely would be very helpful❤❤@@AIAnytime
Hi Kevin, Pls find the GitHub repo for PetCare Chatbot: github.com/AIAnytime/PetCare-AI-Llama-2-Chatbot
@@kevinyuan2735 i need to max 1000pages how to change it?
Thanks for the video. Had a question.. why did you have smaller chunk sizes for the answer splitter compared to the question splitter?
What changes would be needed to code , if we need a Multichoice Question Answer (4 options) format
Good job
Great work❤
What do you do with these kind of warnings:
Number of tokens (667) exceeded maximum context length (512).
Program has this problem discarding many QA pairs because of it. Looks like context window is 512. I tried chunking to smaller chunks but no success.
use this:
def load_llm():
# Load the locally downloaded model here
llm = CTransformers(
model = "mistral-7b-instruct-v0.2.Q4_K_S.gguf",
model_type="mistral",
temperature = 0.3,
config={"max_new_tokens": 2048, "context_length": 4096}
)
return llm
thank you sir it was a great video, your content is so helpful and easy to understand, sir i have small request can you make the chat bot for searching in pdf with highly accurate information in pdf using mistral, langchain, and fastapi?, this will realy help full for us thank so much
Thanks for your comment. Can you check if this video helps: ruclips.net/video/C5mqILmVUEo/видео.html
hi,thanks for your sharing,I have an question about this app,for the size of docs and the number of QAs,do they have limits?
Thanks for the this video, obviously it can help in order to create a data set for fine tuning a model.
my query is what is we have to generate more then 1 contextual question from a single paragraph ,? How to improvise then?
Hello I am using mistral 7b for generating summary. It takes 50 seconds to generate answer could you please tell me some ways to reduce inference time.
I have tried other models too, gptq model and few inference library.
🚀🚀🚀
Thanks Sonu❤
Thanks Sir
Thanks for this video tutorial and your source code.... And, from Q&A data. Can this model continue to generate long-form content based on this data?
Absolutely, it can. I will recommend using Llama2 32k context model in those cases. Infrastructure is something you need to take care of, rest all will be the same.
😊 is there any way to generate complex question with their answer with this model?
how do you prep the pdf files for ingestion... i always get some kinda of encoding error
Can I use this question and answer pairs to make my own dataset for intruct fine tuning right?
Absolutely you can. You can first create QA pair and then add instructions in one column. You can look at Alpaca formatting for instructions.
Works fine.... Thank you
Enjoy
Legend ❤
🎉
Hi can you please share the add_model py for youtube video summarization app. In that github repo that file is missing
Sure, take that file from my other repo. Find it here: github.com/AIAnytime/Haystack-and-Mistral-7B-RAG-Implementation/tree/main
@@AIAnytime Thanks man☺️
hello sir, can you please make a video for deploying medical chatbot in cloud, I really want that.
bhai samjha bhi diya kar kyon varibales bana raha hai aur kyun functions create kar raha hai like its use case...read hi karna tha toh docs hai
How to download the gguf file ?
From the Huggingface. The bloke repo.
Kindly do with gpu as well
Have mentioned how this can be done on the GPU as well. Just pass the GPU layers in CTransformers
@@AIAnytime I tried, but its not utilizing the GPU, please make one video on colab or on ur system, it would be very helpful for me.
Trash tutorial, only works for the pdf provided , for any other pdf there is a token limit exceeding error
legend