haha just did exactly the same thing, had a RAG project that gave me headaches due to bad quality of external knowledge, tried literally every advanced RAG technique and in the end I still had a lot of issues, especially with complex queries. in the end I ended up using smolagents with keyword search, although I had to design a whole separate schema for metadata but this new set up works way better than any of those iterative techniques that try fix the retrieval issues of semantic search. good to know I'm not alone in this😂
This is good! Also, Eagerly awaiting the 3rd part of evals; hopefully u can also show how u are using locally hosted models there - badly needed along with the other items to be released
For very large databases of documents, this is computationally far more efficient than creating embeddings for all of them. For regular text search, it makes more sense to use this approach. However for tables, graphs and illustrations embeddings are maybe still useful even in large datasets. Do you see a hybrid approach likely?
Hi Ronan, very interesting! I'm not at the end of the video yet, but I'm a bit worried about documents in multiple languages (as is usually the case where I live). Even though the agent may convert the essence of the user query to English quite well, if some documents are not in English (Dutch e.g.), then keyword search does not work. I'll watch the video till the end to see if you have a solution to that. Maybe it is even feasible to translate non-English documents and work from there, but that may lead to complexity, performance, cost, and accuracy problems. Or otherwise, translate the query into the language of each document. What are your thoughts on that?
@@TrelisResearch I have not tested yet (but going to), but maybe I was not clear. I did not try to refer to other languages than English but to mixtures of languages (documents as well as queries). This is usually the case in non-native-English countries. Our document bases in the Netherlands typically consist of a mixture of English and Dutch documents and users query in Dutch or English (I do too). Embeddings settle the language gap. I assume this would be a problem when non-semantically searching. Is that true? And do you think translating the query per document would be feasible? I tried with a query in Dutch, but it was automatically translated to English. I guess a way to go is to make the retriever tool ask for a bm25 version of the query in all languages of the documents in the document set, and then match the versions to the documents that are in that language. I'll have a go at it tomorrow.
Yeah you’re right. The timestamps are pretty bad. Thanks for letting me know. I used claude sonnet on a vtt and usually it’s good but this is not. I’ll have to go back to manual I think
![[1hr Talk] Intro to Large Language Models](http://i.ytimg.com/vi/zjkBMFhNj_g/mqdefault.jpg)
haha just did exactly the same thing, had a RAG project that gave me headaches due to bad quality of external knowledge, tried literally every advanced RAG technique and in the end I still had a lot of issues, especially with complex queries. in the end I ended up using smolagents with keyword search, although I had to design a whole separate schema for metadata but this new set up works way better than any of those iterative techniques that try fix the retrieval issues of semantic search. good to know I'm not alone in this😂
Nice
Nice one, Sir. Are you allowedbto share the code functionalities?
Your videos are awesome! Thank you and please keep making these awesome videos!
You’re welcome
Have you tried Docling for markdown conversion of PDFs? I hear it has very good performance too.
Just started reading about it today and will dig more
This is good!
Also, Eagerly awaiting the 3rd part of evals; hopefully u can also show how u are using locally hosted models there - badly needed along with the other items to be released
Actually yeah. Let me do that. Good idea.
Nice! Are you interested in consulting for some exciting projects? Thanks for the vid as always. One love!
Howdy! You can check trelis.com for options - consulting is one of
This is excellent. Thank you.
You’re welcome
Wondering if it would be possible for you to show how to run deep seek locally using hugging face ? Not sure about the compute requirements, either.
Best option is to run one of the distilled Qwen or Llama models - distilled from r1. You can use lmstudio.
Ok probably I’ll do this in the next evals video OR maybe I’ll make a dedicated video.
@@TrelisResearch Bravo !
For very large databases of documents, this is computationally far more efficient than creating embeddings for all of them. For regular text search, it makes more sense to use this approach. However for tables, graphs and illustrations embeddings are maybe still useful even in large datasets. Do you see a hybrid approach likely?
Yeah probably hybrid makes sense in all cases, it’s a matter of time and complexity to code it.
Hi Ronan, very interesting! I'm not at the end of the video yet, but I'm a bit worried about documents in multiple languages (as is usually the case where I live). Even though the agent may convert the essence of the user query to English quite well, if some documents are not in English (Dutch e.g.), then keyword search does not work. I'll watch the video till the end to see if you have a solution to that. Maybe it is even feasible to translate non-English documents and work from there, but that may lead to complexity, performance, cost, and accuracy problems. Or otherwise, translate the query into the language of each document. What are your thoughts on that?
Howdy!
What’s your thinking on why bm25 won’t work?
It should work in any language. There are no embeddings
@@TrelisResearch I have not tested yet (but going to), but maybe I was not clear. I did not try to refer to other languages than English but to mixtures of languages (documents as well as queries). This is usually the case in non-native-English countries. Our document bases in the Netherlands typically consist of a mixture of English and Dutch documents and users query in Dutch or English (I do too). Embeddings settle the language gap. I assume this would be a problem when non-semantically searching. Is that true? And do you think translating the query per document would be feasible?
I tried with a query in Dutch, but it was automatically translated to English. I guess a way to go is to make the retriever tool ask for a bm25 version of the query in all languages of the documents in the document set, and then match the versions to the documents that are in that language. I'll have a go at it tomorrow.
Hello Goat
Note: The timestamps are all wrong. Maybe it’s for some other video. PS love your videos
Yeah you’re right. The timestamps are pretty bad. Thanks for letting me know.
I used claude sonnet on a vtt and usually it’s good but this is not. I’ll have to go back to manual I think
@@TrelisResearchWhisper is a way to go for initial conversion. For longer files Like this, Gemini is best