Great video, dude! Quick question: Does the `upsert_documents()` method save something in the database? After following the tutorial, I see no changes in the created database.
Hey, I love watching your video it is easy to understand and implement. I am struggling in making one project. Can you help me in that by providing ideas or by creating the video. Thanks in advance 👍 Project: Creating the chatbot with opensource ollama model mistral/llama3. To chat with documents [txt,docx,csv,xlsx,pptx,json,pdf]. The output should contain the textual answer along with the url links provided inside the documents. URL link should be provided on the base of the prompt of the user. What did I used: ollama mistral model with chroma DB so the text extracted from the documents will converted into embeddings and stored inside the vector database [chromaDB in my case]. But it was taking time more than 2 hours to store the data from the csv file which have around 1000 rows only. For chat with csv and excel file I used pandasAI, weaviate knowledge graph and neo4j from which neo4j was giving good result but struggles with complex prompt provided by the user. Now I added the langchain agents so break and understand the prompt create the cyper query and shoot it to the neo4j DB. But still not providing output we want even on the simple queries. What I am able to complete: somehow I am able to complete providing the textual answers with url link from the documents [txt, json, pdf, docx]. with around 75 to 85% of accuracy. But still struggle with the csv and excel files.
Hey! Developed similar project just a few weeks ago. Here are few probable solutions - 1. The time taken might be because of the hardware you have. If you are doing this project for fun like me then just bare the time issues for now. But if you are planning to deploy it as a product, go for a virtual instance on cloud services and get some compute power to speed up vector embedding generation. 2. Secondly pdf and txt files can be easily read using UnstructuredPDFLoader from Langchain for document loading and nomic-embed-text from ollama for embedding generation. 3. Again the output generation completely depends on hardware being used but accuracy from this method was higher (at least in my testing) 4. Finally, please check UnstructuredExcelLoader for the excel type data to get proper data out of the sheets.
@@doofsCat Hey! thanks for the help I have used point 2 as you mentioned and talking about unstructuredexcelloader I have used that but it was taking more time my case. And you are right I don't have much better hardware that's fine for now as I am experimenting and it will take time no problem in that. Just looking for better options
@@rushabhshah12200 Hardware is a real pain . I get it too. Only solution is open source services like aws free tiers etc but those expire very fast too. Have a perfect code and test on Google collab, it lasts a bit longer comparatively. Dont waste the resources of collab while writing code. Just use it to test how fast the responses are being generated etc.
Hiiiii sir, I'm your new subscriber.... currently learning DSA from your Playlist.... your explanations are pretty much better than our teachers.... ❤️ from 🇮🇳
Thank you very much. I have already create a postgres DB and add thousand of text in a table like for example : id =1; text =" my text" ....etc. Now if a add postgresML. How can I vectorize it ? Is it done automatically or should I do some script ? BR, Philippe
![Tee Grizzley - Blow for Blow (feat. J. Cole) [Official Video]](http://i.ytimg.com/vi/wNLN5xsx5dc/mqdefault.jpg)
Great video, dude! Quick question: Does the `upsert_documents()` method save something in the database? After following the tutorial, I see no changes in the created database.
is postgres ML a free add on? in the site it is talking about prices 50$ / month , or i am mistaken?
Watching 👀
Can you do a deep dive into their korvus library is that a better solution?
Thx._
Hey, I love watching your video it is easy to understand and implement. I am struggling in making one project. Can you help me in that by providing ideas or by creating the video. Thanks in advance 👍
Project: Creating the chatbot with opensource ollama model mistral/llama3. To chat with documents [txt,docx,csv,xlsx,pptx,json,pdf]. The output should contain the textual answer along with the url links provided inside the documents. URL link should be provided on the base of the prompt of the user.
What did I used: ollama mistral model with chroma DB so the text extracted from the documents will converted into embeddings and stored inside the vector database [chromaDB in my case]. But it was taking time more than 2 hours to store the data from the csv file which have around 1000 rows only.
For chat with csv and excel file I used pandasAI, weaviate knowledge graph and neo4j from which neo4j was giving good result but struggles with complex prompt provided by the user.
Now I added the langchain agents so break and understand the prompt create the cyper query and shoot it to the neo4j DB. But still not providing output we want even on the simple queries.
What I am able to complete: somehow I am able to complete providing the textual answers with url link from the documents [txt, json, pdf, docx]. with around 75 to 85% of accuracy. But still struggle with the csv and excel files.
Hey! Developed similar project just a few weeks ago. Here are few probable solutions -
1. The time taken might be because of the hardware you have. If you are doing this project for fun like me then just bare the time issues for now. But if you are planning to deploy it as a product, go for a virtual instance on cloud services and get some compute power to speed up vector embedding generation.
2. Secondly pdf and txt files can be easily read using UnstructuredPDFLoader from Langchain for document loading and nomic-embed-text from ollama for embedding generation.
3. Again the output generation completely depends on hardware being used but accuracy from this method was higher (at least in my testing)
4. Finally, please check UnstructuredExcelLoader for the excel type data to get proper data out of the sheets.
@@doofsCat Hey! thanks for the help I have used point 2 as you mentioned and talking about unstructuredexcelloader I have used that but it was taking more time my case.
And you are right I don't have much better hardware that's fine for now as I am experimenting and it will take time no problem in that. Just looking for better options
@@rushabhshah12200 Hardware is a real pain . I get it too. Only solution is open source services like aws free tiers etc but those expire very fast too. Have a perfect code and test on Google collab, it lasts a bit longer comparatively. Dont waste the resources of collab while writing code. Just use it to test how fast the responses are being generated etc.
Hiiiii sir, I'm your new subscriber....
currently learning DSA from your Playlist....
your explanations are pretty much better than our teachers....
❤️ from 🇮🇳
do you read comments, I am a big fan I have some questions
Thank you very much.
I have already create a postgres DB and add thousand of text in a table like for example : id =1; text =" my text" ....etc.
Now if a add postgresML. How can I vectorize it ? Is it done automatically or should I do some script ?
BR, Philippe
You are my favorite channel!!!