Thank you for the feedback! Much appreciated! Love to learn and share these things, I understand best by building and tinkering. Some more cool RAG stuff coming up soon! :)
Thank you! It seems RAG usecases are in demand right now, and there's also uses for own internal tools. Some of these of course will appear as part of available tooling, but being able to do it yourself is great for like-minded tinkerers that like to understand things by building them, I think.
Create a Basic RAG model AI chatbot using any open-source model available. Database: Zilliz/Supabase or any Vector DB, of your choice. Backend: Java Frontend: HTML/CSS/JS (or anything you are familiar with) Backend should haveAPI for: • Upload a file or text - text should then be converted into chunks and then embeddings (You can use any open-source embedding model or paid one too). Embedding should then be stored in a vector DB with vector index of your choice (Cosine/ L2/KNN). • Chat API -> prompt or user queries should be passed in this API and using prompt engineering response should be derived from any Language model. Frontend: UI does not have to look pretty. basic pages should be there - 1. For uploading docs or text for training the RAG model. 2. Basic chat interface where user can send message and receive the response. Sir please help me how to do this project🙏🙏🙏🙏🙏🙏
Hi! Thanks for sharing ! Very useful! Is it correct to use this idea to read .pdf invoices? I have a situation that I need to read .pdf invoices and extract the information into JSON format. What is most correct approach? Extracting text from the .pdf and send this text to the LLM or use this RAG idea? Thanks.
Hi, thank you! I would not include RAG easily in anything that touches money, like invoices - except if you just want to get some summary information with flexible queries. Too easy to get some zeros or decimals wrong there. But depends entirely on your usecases. To read pdfs in depends on their format, one approach is to use similar code as I'm using here. Some pdfs are images with no structural content that could be extracted, but I suspect with invoices this should work.
@@DevXplaining thanks for your response. Actually my usecase is not that critical, I just need to read the information from the pdf invoice and exhibit them on a HTML form. The user will edit the information and change it if necessary. I feel like I don´t need RAG for this but I am not sure. Maybe if I only extract the text from the pdf and send this text and the empty JSON to be fill up by the LLM will do.
Thank you! I did not include frontend because on top of API that is quite straightforward, and tried to keep the video compact. But I've been thinking of doing a bigger video with full stack, because it looks a lot nicer. But not there yet, at least. It would really be a trivial NextJs (or any other framework) web app that you dump into spring boot 'public' folder under sourcecode. So I'll definitely take that under consideration, but probably not going to happen soon.
one doubt i have ollama which will have a ai model lets say mistral or meta now i am connecting this to my springboot app and lets say i have a pdf about my product can show me how a rag implementation done here and when a api is sent it should give me the data about the PDF
Hi, thanks for your question! Short answer is yes. Longer answer is: Each LLM has a context window: Number of tokens it may take as input. That can be roughly translated to how many much text it can eat. So for a runtime solution, with modern LLMs, typically a lot of text or even contents of a single file can fit into the context window, and thus there would actually be no need to involve RAG or a vector database like here. Just include the file contents/text along with any questions you would like to ask, simple as that. The RAG/Vector database approach comes in useful for two scenarios: 1) We have a lot of files, or a book, or books, and would like to ask questions about all of them. And 2) we would like to improve the quality of the answers by NOT including a lot of text, but instead just including relevant text we have retrieved that most likely is necessary to answer that question. So yes, you can pass in a file via API or UI, somehow parse its contents to something resembling structure, and include it in the LLM call. Or, you start keeping a database of files, and set them up so it's easy for you to search and find the relevant chunks.
Yes, this video is a bit outdated now that Spring AI is at 1.0 and out there, things have changed and as you guessed, EmbeddingClient is gone. Thanks for the heads-up! I should probably revisit this topic, perhaps redo with current version, and dig a bit deeper, but meanwhile, here's up-to-date documentation on the 1.0 approach: docs.spring.io/spring-ai/reference/api/embeddings.html#_api_overview Not a lot of changes, but move to EmbeddingModel interface instead. Here is an example how to use that one (for OpenAI): docs.spring.io/spring-ai/reference/api/embeddings/openai-embeddings.html
@@DevXplaining thank you so much! I will try to implement it. I tend to have difficulties learning from documentation only. Any advice? Do you work as a mentor maybe?
Hi, yes, that's a known issue, it's not perfect. Pdf readers often struggle with table data even in the best RAG solutions, and of course cannot easily understand any information that's not in text format already. It's mostly for poc level example, to make it work better, much more work is needed. What can be done? - check which part is not parsed properly - you can isolate the code that parses pdf and play with it - there are typically adjustments you can make to better meet your needs, but as mentioned, it can mostly handle text chapters and titles, table data and images need some other approach overall - possibility to see the library issues list, see if anyone else is struggling with same issues - possibility to check to see if there's a better library available for your needs - there typically are options. This was just one that was fast to get working. So, pdf documents are not the easiest usecase to feed for RAG, something that would be better structured is typically going to give better results. But it's a nice starting place. I'm a bit concerned myself about losing the graphs, images, and potentially having table data cut in halves etc, those can alter the results vastly. If you can find any better means of converting the pdf to any structured format that would retain what you need, that would improve the results.
Gratitude for your time and energy! Thank you for sharing such informative content!
Thank you! Much appreciated!
Nice and well structured explanation
Thank You, much appreciated!
Thank you ! Very nice!
Thank you!
Greate video with lots and lots of great information.
Thank you for the feedback! Much appreciated!
Love to learn and share these things, I understand best by building and tinkering. Some more cool RAG stuff coming up soon! :)
good ones , we want more usecases for AI and springboot
very good video, thanks for the into, great more examples about rag.
Thank you! It seems RAG usecases are in demand right now, and there's also uses for own internal tools. Some of these of course will appear as part of available tooling, but being able to do it yourself is great for like-minded tinkerers that like to understand things by building them, I think.
Create a Basic RAG model AI chatbot using any open-source model available.
Database: Zilliz/Supabase or any Vector DB, of your choice.
Backend: Java
Frontend: HTML/CSS/JS (or anything you are familiar with)
Backend should haveAPI for:
• Upload a file or text - text should then be converted into chunks and then embeddings
(You can use any open-source embedding model or paid one too). Embedding should
then be stored in a vector DB with vector index of your choice (Cosine/ L2/KNN).
• Chat API -> prompt or user queries should be passed in this API and using prompt
engineering response should be derived from any Language model.
Frontend:
UI does not have to look pretty. basic pages should be there -
1. For uploading docs or text for training the RAG model.
2. Basic chat interface where user can send message and receive the response.
Sir please help me how to do this project🙏🙏🙏🙏🙏🙏
Awesome idea! Working on something else right now, but this sounds like a good plan for a long form video, and a fun one.
Give me your job, Ill do it for easy
Hi! Thanks for sharing ! Very useful! Is it correct to use this idea to read .pdf invoices? I have a situation that I need to read .pdf invoices and extract the information into JSON format. What is most correct approach? Extracting text from the .pdf and send this text to the LLM or use this RAG idea? Thanks.
Hi, thank you!
I would not include RAG easily in anything that touches money, like invoices - except if you just want to get some summary information with flexible queries. Too easy to get some zeros or decimals wrong there. But depends entirely on your usecases.
To read pdfs in depends on their format, one approach is to use similar code as I'm using here. Some pdfs are images with no structural content that could be extracted, but I suspect with invoices this should work.
@@DevXplaining thanks for your response. Actually my usecase is not that critical, I just need to read the information from the pdf invoice and exhibit them on a HTML form. The user will edit the information and change it if necessary. I feel like I don´t need RAG for this but I am not sure. Maybe if I only extract the text from the pdf and send this text and the empty JSON to be fill up by the LLM will do.
Nice video! Can you do it with front end like nextJs ? it will be really appreciated and helpful!
Thank you! I did not include frontend because on top of API that is quite straightforward, and tried to keep the video compact. But I've been thinking of doing a bigger video with full stack, because it looks a lot nicer. But not there yet, at least. It would really be a trivial NextJs (or any other framework) web app that you dump into spring boot 'public' folder under sourcecode.
So I'll definitely take that under consideration, but probably not going to happen soon.
Kiitos paljon
one doubt i have ollama which will have a ai model lets say mistral or meta now i am connecting this to my springboot app and lets say i have a pdf about my product can show me how a rag implementation done here and when a api is sent it should give me the data about the PDF
Is it possible to handle the file at runtime, where the user provides the file via an API, and then chat based on the content of that file?
Hi, thanks for your question! Short answer is yes. Longer answer is:
Each LLM has a context window: Number of tokens it may take as input. That can be roughly translated to how many much text it can eat. So for a runtime solution, with modern LLMs, typically a lot of text or even contents of a single file can fit into the context window, and thus there would actually be no need to involve RAG or a vector database like here. Just include the file contents/text along with any questions you would like to ask, simple as that.
The RAG/Vector database approach comes in useful for two scenarios: 1) We have a lot of files, or a book, or books, and would like to ask questions about all of them. And 2) we would like to improve the quality of the answers by NOT including a lot of text, but instead just including relevant text we have retrieved that most likely is necessary to answer that question.
So yes, you can pass in a file via API or UI, somehow parse its contents to something resembling structure, and include it in the LLM call. Or, you start keeping a database of files, and set them up so it's easy for you to search and find the relevant chunks.
I don't mean to mock you, but when you say "model", it sounds like "module" in my ears. 😅
nice video bro, keep create content like this...
Haha, I actually aporeciate feedback on my pronounciation, lets me work on it. Thank you!
It doesnt appear that EmbeddingClient exist. Am I missing some dependency or is it deprecated?
Yes, this video is a bit outdated now that Spring AI is at 1.0 and out there, things have changed and as you guessed, EmbeddingClient is gone. Thanks for the heads-up!
I should probably revisit this topic, perhaps redo with current version, and dig a bit deeper, but meanwhile, here's up-to-date documentation on the 1.0 approach:
docs.spring.io/spring-ai/reference/api/embeddings.html#_api_overview
Not a lot of changes, but move to EmbeddingModel interface instead. Here is an example how to use that one (for OpenAI):
docs.spring.io/spring-ai/reference/api/embeddings/openai-embeddings.html
@@DevXplaining thank you so much! I will try to implement it.
I tend to have difficulties learning from documentation only. Any advice? Do you work as a mentor maybe?
Hi I tried this RAG with pdf document , some of pdf is not parsed by PagePdfDocumentReader
Hi, yes, that's a known issue, it's not perfect. Pdf readers often struggle with table data even in the best RAG solutions, and of course cannot easily understand any information that's not in text format already. It's mostly for poc level example, to make it work better, much more work is needed.
What can be done?
- check which part is not parsed properly - you can isolate the code that parses pdf and play with it
- there are typically adjustments you can make to better meet your needs, but as mentioned, it can mostly handle text chapters and titles, table data and images need some other approach overall
- possibility to see the library issues list, see if anyone else is struggling with same issues
- possibility to check to see if there's a better library available for your needs - there typically are options. This was just one that was fast to get working.
So, pdf documents are not the easiest usecase to feed for RAG, something that would be better structured is typically going to give better results. But it's a nice starting place. I'm a bit concerned myself about losing the graphs, images, and potentially having table data cut in halves etc, those can alter the results vastly. If you can find any better means of converting the pdf to any structured format that would retain what you need, that would improve the results.
Good video but, editor color does not match for broadcast and dragging mouse in code editor is not good
Thank you! I'll take a look at those, details are important.
please create from the scratch
Noted, need to make a longer video then but would be fun :)