Thank you so much for pointing this out. I am running a RAG application in production system. The quality of documents I work with is not that great. I have been asked to improve the accuracy of whole RAG pipeline. Hence this is very helpful. :)
I appreciate these videos. I'm still trying to get this all figured out. I have a new system, and a big reason to get it, is to run local models I can share out with colleagues. (Keep it all local).
Is anyone adding the overlap, when split by paragraph, to be the last sentence (conclusions) of the ptevious paragraph and the first sentence (introduction/continuation) of the next paragraph? Also, metadata should be keywords produced by a very small local llm, and then you can make a knowledge graph of the keywords.
Thank you for sharing the video. I have a query regarding a PDF document containing numerous tables. I am currently developing a RAG system, and I am encountering challenges in extracting information from the tables using standard PDF loaders. I have explored using GPT-4 on images, which proved successfully, I asked it to extract it using json form and it worked, but I am seeking an automated solution. Could you kindly suggest effective methods to enhance table content extraction ?.
It seems that not only do you ensure your content context is maintained, but you should also see a more economical parsing of the ingested text. Is that correct?
Thanks bro! I can't wait to take your RAG course. Btw, here's a great tutorial on evaluating 8 different RAG models. It show a nice comparative analysis of different metrics of different RAG techniques: ruclips.net/video/nze2ZFj7FCk/видео.htmlsi=NzgKUeUlTW9ZYn00
Hi. Thanks for your videos. I remember your videos using normally a chunk size = 1000 and overlapping = 200 characters. That was for ChatGPT, LLama?, Mixtral? or others. What is your recommendation size and overlapping for these very well known LLMs?
If you are interested in learning more about how to build robust RAG applications, check out this course: prompt-s-site.thinkific.com/courses/rag
Thank you so much for pointing this out. I am running a RAG application in production system. The quality of documents I work with is not that great. I have been asked to improve the accuracy of whole RAG pipeline. Hence this is very helpful. :)
Very informative video, ❤ your style of explanation. Keep sharing more on this topic
I appreciate these videos. I'm still trying to get this all figured out. I have a new system, and a big reason to get it, is to run local models I can share out with colleagues. (Keep it all local).
If you are interested in leanring more about Advanced RAG Course, signup here: tally.so/r/3y9bb0
Very nice! I do hope you get into some advanced prompting, though. Good prompting can make a huge difference with RAG.
Did you evaluated also Self-RAG or CRAG or GraphRAG or SubDocument-RAG(Summarizing)? To improve answer quality.
Is anyone adding the overlap, when split by paragraph, to be the last sentence (conclusions) of the ptevious paragraph and the first sentence (introduction/continuation) of the next paragraph? Also, metadata should be keywords produced by a very small local llm, and then you can make a knowledge graph of the keywords.
Thank you for sharing the video. I have a query regarding a PDF document containing numerous tables. I am currently developing a RAG system, and I am encountering challenges in extracting information from the tables using standard PDF loaders. I have explored using GPT-4 on images, which proved successfully, I asked it to extract it using json form and it worked, but I am seeking an automated solution. Could you kindly suggest effective methods to enhance table content extraction ?.
It seems that not only do you ensure your content context is maintained, but you should also see a more economical parsing of the ingested text. Is that correct?
Yes please more. Thanks
Aprreciate the video! Very interesting
Thanks bro! I can't wait to take your RAG course. Btw, here's a great tutorial on evaluating 8 different RAG models. It show a nice comparative analysis of different metrics of different RAG techniques:
ruclips.net/video/nze2ZFj7FCk/видео.htmlsi=NzgKUeUlTW9ZYn00
Best chunking is using agentic chunking with grouping, but it costs money
Not if you are using a local llm as agent
@@maxlgemeinderat9202interesting, thanks
Hi. Thanks for your videos. I remember your videos using normally a chunk size = 1000 and overlapping = 200 characters. That was for ChatGPT, LLama?, Mixtral? or others. What is your recommendation size and overlapping for these very well known LLMs?
Can you make a video comparing how chunk size is a trade-off between accuracy and recall?