"I want Llama3 to perform 10x with my private knowledge" - Local Agentic RAG w/ llama3
HTML-код
- Опубликовано: 22 май 2024
- Advanced RAG 101 - build agentic RAG with llama3
Get free HubSpot report of how AI is redefining startup GTM strategy: clickhubspot.com/4hx
🔗 Links
- Follow me on twitter: / jasonzhou1993
- Join my AI email list: www.ai-jason.com/
- My discord: / discord
- Corrective RAG agent: github.com/langchain-ai/langg...
- LlamaParse: github.com/run-llama/llama_parse
- Firecrawl: www.firecrawl.dev/
- Jerry Liu build production-ready RAG: • Building Production-Re...
⏱️ Timestamps
0:00 Intro
1:33 How to give LLM knowledge
3:05 Problem with simple RAG
5:55 Better Parser
9:01 Chunk size
11:40 Rerank
12:39 Hybrid search
13:10 Agentic RAG - Query translation
14:35 Agentic RAG - metadata filtering
15:52 Agentic RAG - Corrective RAG agent
17:33 Install LLama3
18:00 Code walkthrough
👋🏻 About Me
My name is Jason Zhou, a product designer who shares interesting AI experiments & products. Email me if you need help building AI apps! ask@ai-jason.com
#llama3 #rag #llamaparse #llamaindex #gpt5 #autogen #gpt4 #autogpt #ai #artificialintelligence #tutorial #stepbystep #openai #llm #chatgpt #largelanguagemodels #largelanguagemodel #bestaiagent #chatgpt #agentgpt #agent #babyagi 
Наука
This is prob one of the best RAG video I've seen, so many learnings in 20 mins
One of the most informative RAG videos I’ve seen. Can’t wait to see more from your channel.
Man, your videos keep getting better every time I look. You have a great mind and your presentation is excellent. Thank you very much, again, for sharing!
he is much better than 99.9% wanna be over hyped ai gurus on youtubu, twitter and linkedin!
Yet again an amazing tutorial, thanks so much Jason!
This is the best RAG video on the internet, awesome job, no fluff, high complexity but easy to understand, nice work
Holy crap! This gave me such amazing background knowledge, love it! Now, what would be extra cool, would be if you could do a real "hands-on" type of workshop to go through it all by setting up the environment completely, including the actual training/RAG implementation of a set of various document types (PDF, excel, website etc..) to extend a locally running llama 3 instance 😊
Great content! Thanks for putting in the effort. Will use this.
00:05 AI can revolutionize Knowledge Management
01:46 Llama3 can process precise knowledge with fast inference
05:27 Market strategy for AI startups
07:16 Convert PDF files to markdown format for enhanced accuracy and control
10:47 Finding the optimal chunk size through experiments
12:34 Hybrid search combines Vector search and keyword search for better results
16:12 Building a local agentic RAG with llama3
17:48 Running Llama3 model on local machine and using Visual Studio Code
20:53 Setting up key components for Llama3 performance
22:20 Creating a complex agentic RAG workflow for document retrieval and answering
1) The link for the corrective RAG agent had an extra URL attached at the end which caused it to fail; manually tracing the link got me to the proper location
2) LlamaParse looks like a wonderful tool, since I have a lot of documents with equations, and I really need it to grab equations, if for no other reason than to return them. Unfortunately, LlamaParse requires an API key and seems to send PDFs off for processing, something that others have noted and there is an open issue from 2 weeks ago. As of 3 hours ago, it's still an open issue - clearly most companies don't want to send internal docs out of house. Hopefully this gets resolved soon.
3) Really liked your presentation - easy to follow every step with the provided materials.
Hopefully we will have more better options for local use - shame it's not a local only pipeline yet
yes - I have found this issue too. LlamaParse seems use OpenAI llm to process the pdf and it leads to the privacy concerns.
You always amaze me by the amount of knowledge I get from your videos
What a great video! Thanks for sharing your knowledge
Really great tutorial, teaches a lot in very short time! Thanks!
Excellent video! I don't have much experience with RAG and this was sooo helpful!
Firecrawl boosted our RAG accuracy at our company. fast + provided good markdown format.
Llama parse also super helpful too! Amazing video Jason! This is gold!
Edit: thanks for the likes :)
The search api is just insane on firecrawl
dude... great video! Thanks for the knowledge!
Jason, I watch a lot of AI videos but I learn the most from yours. I am actually excited everytime i see you have put another one out. Keep up the great work!
Awesome content!
Didn't know about the Agentic RAG techniques, thanks for sharing!! That's definitely a trade off between speed & quality, but good to have the option
This is extremely helpful! Awesome!
You're the man 💯👏
right when i needed it, thank you man!
also, just finished watching and i understood the theory behind it but kinda got lost during the code explanation, i might watching again and again
Solid video Jason
Amazing tutorial! Thank you
Great tutorial! Thank you
Amazing info shared -. Thank you!
Keep this up. This answered to loads of questions I have had previously, and were not answered in any of the HuggingFace tutorials!
Great timing! Why do you always read my mind JASON!!?! lol
many many thanks, bro!
Subscribed, dont have an AI company since I'm still a poor student... this video was very informative, the man speaks at two times speed just like my professor. I respect it 😁
wow nice work thanks!
Thanks!
awesome jason thank you
Thanks... Awesome video
Great video, thanks
great video keep making these please.. only "criticism" / advice if you can call if that is to keep things focused on local / open source solutions as much as possible.. love the use of Ollama here for example.. things that perhaps don't require API keys, subscriptions, external integrations / dependencies help people like me understand more of what's going on in a workflow like this! thanks again!
This is a great trick, thanks
I am literally using this technique now in my internship for a project. I went through so many approaches and ended up on my version of this one. Wish you released this video about 2 months ago lol
You are relevant, Subscribing to your channel!
Such a bait and switch. Thumbnail promises fine tuning tutorial. Delivers best improve-your-RAG video on the internet. Excellent work.
OG Jin Yang from Silicon Valley.. Amazing video 🎉
Amazing lesson! I learned a lot in just 20 min!
Thanks! It's so fascinating how these programs 'think.' Even if I don't install one, concepts like chunking seem to translate to humans as well.
Platform agnostic LLM space overview videos from Jason are the best on AI YT
I prefer finetuning to RAG first then RAG on top of the finetuned model. Just a simple QLORA is all you need. It really helps a ton.
How would you go about doing that, as in just do it backwards from the video?
The corrective RAG schema explains why AI often tries to bring results from the web even when you tell them not to in prompt. If it doesn't understand the source properly it will look elsewhere. This was insightful, thank you.
This is epic! keep up...
Best video💯
Clicked that BELL too! 🔔
great video jason! quick question, im wondering if a knowledge graph in place of vector database would be better since it mitigates the lost in the middle problem?
I have to say, it is great :D
This answer a lot of questions why my chat with PDF doesn't work, llama parser & firecrawl looks so freaking good!
You got a sub. Finally, an AI channel that actually teaches.
Thanks Jason, great video, this explains RAG pretty well. Subscribed!
I thought we were gonna fine tune llama3 😢 but the fire crawl implementation looks unreal I’ll have to check that out and add it to my rags.
I don’t know how well it’ll work for RAGs but people have extended the context window like crazy and still can do the needle in haystack to around 130k.
If you have 64gb on the Mac you can try out the 256k context window Llama 3 released by Eric Hartford. Would love to see a side by side with both of them using the same embeddings.
Great video, thanks. New subscriber (and like) here. I had a couple of questions though: why use langchain? It seems unnecessary from what I have read. Would also love a demo ipynb/copy of code.
Awesome content Jason. A Question. I need to create an AI psychologist and store college data, but this college data is a guide of what to speak, not the content itself.
In that case, what is the best approach, RAG or Fine-tuning?
Hi brilliant session , do you have a link for the notebook ?
thx
here come dat boi!!!!!!
Thank you. Can you say a little about your hardware setup for this work? This information is missing from a lot of online sources.
would be great to get a video on best methods for data extraction from these pdfs
The speaker in the transcript discusses the use of AI, particularly large language models, in knowledge management. They highlight that AI can provide value in managing vast amounts of documentation and meeting notes, which can be overwhelming for humans to process. The speaker also mentions the potential disruption of traditional search engines like Google by large language models, which can provide hyper-personalized answers based on their extensive knowledge.
The speaker then introduces the concept of a retrieval augmented generation (RAG) pipeline, which involves extracting information from real data sources, converting them into a vector database, and retrieving relevant information to answer user queries. However, they also note the challenges in building a production-ready RAG application, including dealing with messy real-world data, accurately retrieving relevant information, and handling complex queries that may involve multiple data sources.
The speaker also discusses various tactics to mitigate these challenges, such as better data preprocessing, optimal chunk size, relevance-based retrieval, and hybrid search methods. They also mention the use of agentic RAG, which utilizes agents' dynamic and reasoning abilities to decide the optimal RAG pipeline and improve the answer quality.
The speaker concludes by expressing their curiosity about how AI-native startups operate and embed AI into their business processes. They recommend a research document on the subject for those interested.
In summary, the speaker's points are:
1. AI, particularly large language models, can provide significant value in knowledge management.
2. Traditional search engines could potentially be disrupted by large language models.
3. Retrieval augmented generation (RAG) pipelines can be used to answer user queries based on private knowledge.
4. Building a production-ready RAG application is complex due to challenges like messy real-world data, accurate retrieval of relevant information, and handling complex queries.
5. Various tactics can mitigate these challenges, including better data preprocessing, optimal chunk size, relevance-based retrieval, and hybrid search methods.
6. Agentic RAG can further improve answer quality by utilizing agents' dynamic and reasoning abilities.
7. The speaker is interested in how AI-native startups operate and embed AI into their business processes, and recommends a research document on the subject.
Dead internet thory is getting closer and closer every day
What about preparing data, for exemple as question / response, the response would be used to generate embedding and the response would be the data retrieved ?
Great video Jason, I only missed routing as a technique to determine if your question should really go through the RAG. James Briggs has done a few good videos on “semantic routing”.
Is your example notebook available somewhere?
I'm wondering the same thing. Don't see a link to a github repo
Hey Jason thanks for the video, I think it helps a lot. Can I apply on GPT as well?
Goddamn it Jian Yang
Thanks, Jason, incredible as always! Would you consider sharing the code from the walkthrough? 🙏
Thanks mate, appreciate it! Code is in the description link!
@@AIJasonZ Link is not there
Amazinnnnggggg🎉🎉🎉🎉
Great video. How do I add PDF documents and llama_parse to the python notebook?
Interesting. Someone needs to create a wrapper which works out the best way to answer questions / queries, based on the input and question/query. I think intelligence of system could then be increased.
great content! why wouldn't you use groq to speed up the agent response?
Very usefull, thank you! Is it posible for the model to retrieve images or graphs from a PDF, or it's only text?
Great thanks. Can we get the repo and link to the colab notebook?
too many api calls here - do it local with no api calls - better and the model has to be able to crawl more doc formats - people will probably do p2p, real time and uncensored models for 'real' open source ai that has no limiting factors like api calls or tokens - this is where things need to go in order to take off, gain relevance and leverage economies of scale, of course cxl and better i/o will help but those are on the way. real open source ai will hit smb mkt in about 4-5 years and there will be more innovation and discovery - exciting times as we all watch the development curve
Good rag video, the thumbnail taking about "training llama3" is hurting my brain tho
I don’t understand everything but I can feel the gold penetrating my ears
amazing as always. could you share the notebook please
I watch lots of AI videos and 99% of them are just a waste of time. As an AI engineer, this channel is hands down the BEST yet
KEEP UP👏🏼
I'm a simple man. I see a new AI Jason video, I click.
We been trying to build a middleware that connects with any inventory ERP to be able to have real time data information about inventory data for the chatbot
Came to train my 3 Llamas... Now I'm a full stack developer.
Hi, what are the areas current LLMs excel at?
I am new to this world of AI, but not IT (familiar with infra). It is good that people are trying out things to see what it can do. But my naïve thoughts are that as a language tool, it just looks for patterns of words that appear close together, and knows enough of the formation of language that it produces text that is not only readable, but also relevant. But this surely must have limits, if it does not actually understand?
Would it be serving up answers from a well vetted and written sources such as internal KMS by using this RAG method? Our team was thinking about it use for education / learning - perhaps tied into custom flashcard and evaluation of human provided answers. Alongside the still very useful text summarisation, alternative wording suggestions.
I like using gemini for getting quick up to date answers, and chat gpt for stuff that doesn't require up to date stuff
thanks Jason, can i use llama on API and train PDf files in a specify directory train to respond
Great video. Thanks! A lot of very good tips!
Curious how this workflow changes with bigger context length. Gradient just released Llama-3 8B with a 1M context length
great video! Is there a github location with the code?
Would you have plans to create a tutorial that connects what ur teaching here and running thing on something like AnythingLLM that allows document reading to create embeddings.
How would you go about parsing documents of all kinds of types? PDFs, Excels, Word etc...Is there a way to achieve this with only one parser? Or how would you go on about this issue?
4:36 Someone walks into the void and disappears
damn real life Jian Yang
Is those steps and advices are explained on your website ? It would be amazing if you could share the code 😮
Can we also finetune the 70B model? Even if its not local
Would there be a way to automate this with Obsidian? I sporadically log everything in Obsidian and it would be amazing to find a way to do this with Obsidian
There is a problem with the "Corrective RAG agent" URL in the description.
Damn it Jin yiang 😂
👍👍
Can you share the code in the video?
I was hoping that was the case since it's a "simple" workflow
The code is personal you need to apply for a download link with meta and it will provide the code to copy / paste
@@pollywops9242 apply where? I don’t see it
@@pollywops9242😂
Fucking dope bra
Can u create end to end custome fine tuning LLM LLAMA with API