Just found out your channel and subbed after watching this video. Very good quality video! Keep up the great content creation! I am impressed with your motivation to answer the questions from your comment section. Not every youtube channel cares about answering subscriber questions and doing content around what people comment , so very good job!
Make sure you have installed all the frameworks listed in the requirements.txt file. Rewind the video, he explains how to do it all at once. pip install -r requirements.txt If you have done that, make sure you are working in the correct environment. The same environment you installed the python frameworks in. He explains that as well.
Would like to see ollama with openai assistant or similar that can use a file and have assistants do action to the file such as execute code or clean up data etc.
Actually using it in linux platform but still I gave the command ollama serve, I'm getting no GPU detected, Please give solution for this ! @@PromptEngineer48
Followed your instructions here and installed on Raspberry Pi4. Works but of course painfully slow and chip approach 145 degrees F which slows things down as well. But it works and may try on Pi5. Was using a pdf manual for Viking drill press for document. Have to try something with just text.
If I delete the cache folder name db , will this effect how the program will ingest the files I supply or will create a new db and then ingest the new files?
About the context window. I have noticed that it cannot exceed over 2k tokens even though Mistral can support up to 8k. From what I have tested so for, it is like the bot identify itself as a GPT-3, is it because of the openai library?
yes. That is because we start everything compatible with OpenAI API. then we shift to opensource APIs. We could instead work for OpenSource APIs for the start. 😁
My brain is exploding because I was just using your code repository earlier today and then closed everything out. I'm trying to run it again and I'm now getting a bunch of depreciation LangChain warnings. I still get a response but the response is no longer referencing or citing the documents I placed in the folder. Any idea why this would happen?
Great video!! However if i follow the instructions my results are different. I created the source_documents folder an put in another pdf-file. When i then execute the "python3 ingest.py" the ingestion seems to work fine. But when i afterwards exceute the privateGPT.py and start to interact with the llm it still responses to "Think and Grow Rich"-Book.
@@PromptEngineer48 can we not append the pdf files, do we have to keep removing them. when a new file is added and ingested, it should add the document into its cache and should be able to response from both the document isnt it
Thanks for doing this: It seems like when one clones the repository, you need to delete everything in the db folder or it will mess up the results of the information you are trying input yourself.
Does anyone get this warning - UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown (and subsequent crash)
Thank you very much, all the videos you have made are great! However, when I was building the local llm, I found that there are multiple frameworks, such as privateGPT, localGPT, langchain, etc. Similarly, I found that there are very many choices for the llm as well as the vector database (e.g., hugging face vs. ollama), which gave me a big headache, and I was wondering if you could make a I was wondering if you could make a video that explains your recommendations for each part of the process of building a rag-based personal local document chat llm?
That would be a very good video but very less relevant as everyday we have so many updates. But this is something I can create. thanks for the idea. Definelty will work on that
@@PromptEngineer48 Thank you for your response! It's true that, as you say, the various programs are moving so fast. Since I'm hoping to be able to build something myself via langchain, I'm starting to work based on your github project, and some of the other out-of-the-box projects (e.g. open-webui, privateGPT, etc.) are just too heavy for me to get into and modify.
Great Tutorial, Why is it saying loading 235 new documents when it has only one in the source_documents folder. I had 2 in mine and it said 8, wondering why.
thank you, sorry but i don't see the old chat interface :-) i have 2 questions how can i remove a ingested document that i don't need? ist it possible to chat with the docker interface ? or is it only for terminal??
1. U can remove the file. Then delete the db folder and __pycache__ folder.. then run python ingest.py again and python privateGPT.py again. 2. Web UI interface is not integrated here right now. But that is some ok my pipe line as well. I am working on that.
Thank you for this great content! I was able to get this working on my M1 Mac. I was able to run `python ingest.py` and the `python privtagpt.py` commands. However, when I asked it to summarize the document I had uploaded, it referenced the "Think and Grow Rich" document that you showed in the video, rather than the test document I uploaded to the source_documents folder. How do I clear out the embeddings from the "Think and Grow Rich" document, or clear the chroma db database embeddings completely before running the ingest.py command again?
Hi, while setting up Virtual environment I'm getting an error, 7:39 what programs do I need? I have installed Miniconda and it's still no running the line. Thks
Wonderful, thank you! My dream would be to feed my local model with all my books in epub or pdf format just once and the model will keep a memory of those. Then from there I have so many ideas but not sure if feasible, maybe someone can help? 1) for each book create a mind map of concepts 2) a diagram of how each book is related to the others (citations, same authors, same topic, related concepts) 3) given a question or a topic the system can point me to which book is better to read. Probably impossible at the moment, isn't it?
@@PromptEngineer48 Awesome! Another one could be to automatically find the book category. Since the system has already ingested all the books it already knows the discussed topics in each book and from there it can assign each to an ontology of category and subcategories. Like non-fiction/self-help, fiction/novel, non-fiction/self-help/personal growth, and so on. I have other 100 of these ideas, ping me if you need more :D
I would also be interested in something like this if possible as it would be amazing to do this in a repository of books that you know. If you could do this with a non-trivial number of books, say, 100-150 (from Project Gutenberg), it would be a great application of local LLMs.
there are research going on. you need to find it from huggingface or ollama .. see the benchmarks and decide which model you want to use. since we have so many options, I cannot name one. but it's a case to case basis.
@@PromptEngineer48 even after ingesting the PDFs,I am getting answers for some questions that are not in pdf.Is it because of the already trained model?
I'm getting CondaError: Run 'conda init' before 'conda activate' during my installation. I did try conda init but then it says "no action taken". As usual I can't get a step-by-step tutorial to work.
Although I don't get the (base) in front of my path like you did after Preparing transaction: done Verifying transaction: done Executing transaction: done
Hello, i keep getting error ERROR: Could not find a version that satisfies the requirement onnxruntime>=1.14.1 (from chromadb) (from versions: none) ERROR: No matching distribution found for onnxruntime>=1.14.1
I had the error for ChromaDB too. If you Google for them you will probably find an answer. For ChromaDB this solved it for me: - sudo apt install python3-dev - sudo apt-get install build-essential -y
Thanks for your tutorial. I am trying this on Windows pc. I am in anaconda prompt. I am stuck at this error "Error: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based project". I tried installing installing visual studio build tools but the error still persists. Any idea?
I can not update the db, when I ask the agent about the new added document, it still gives answers about the document in this video. It is kind of confusing.
@@PromptEngineer48ok managed to get Conda working. However when I run the python3 ingest.py, I got an error: line6, in from tqdm import tqdm. ModuleNotFoundError: No module named ‘tqdm’
Ok made more progress but now stuck in pymupd. Tried installing it but keep getting message Requirement already satisfied: /opt/anaconda3/envs/privategpt/lib/python3.11/site-packages
Thanks, that’s a great video! I’m testing privategpt for some time and I would love to know if you’re experiencing hallucinations from the chat? And yes mistral seems to be a good model but Zephyr and Dolphin seems to give better answers with a little less performance, depending on the context. Therefore, I couldn’t Get rid of some hallucinations, I would say the reliability of information is 45-65%
Hello @PromptEngineer48 , I new to python and want to learn this. I followed your instruction in your great video, but had many errors when installing modules in requirements.txt. Any update?
Very interesting, easy to follow. I tested this with a music instrument manual, and it wasn't giving accurate answers at all. Any ideas on how to improve this? It's unusable for this type of document. Makes me wonder how accurate it would be with any content. Thank you!
This has been very instructive, thanks! Is there an LLM that's better than Mistral at working with very technical documents, i.e. lengthy API implementation documents? I set up my environment exactly as instructed, got my docs indexes and could get some answers on them. But things get murky when I go very specific, with incorrect or incomplete answers.
Awesome idea. Now use Streamlit or Flask as GUI and the solution is perfect. Thanks for the inspiration. Questions: 1. which OS are you using? 2. python version? 3. do you rather use CPU or GPU? Would an M3 with 128 GB also be sufficient for a quick training / fine tuning of hugging face models up to 20B? Thanks for the answer in advance. Hp
1. Mac OS. Pretty basic. Macbook Air M2, 8 GB 2. python version 3.9 ++ version 3. M3 with 128 GB.. Hmm when you say fine tuning it depends on the model. But here is a rough calculation. If you have a 20B parameter model in 32 bits, then you need 20x32/8 = 80 GB GPU. your system should be able to do the fine tuning. Else please go for 4 bit quantization, then the requirement will reduce by 8 times, now requiring only 80/8 = 10 GB of GPU.
How would you do this if you have ollama in docker? Or even cloud gpu like runpod? Or Linode? Seems like a good video for the future as you explain,and teach better than a majority of the creators.
Okay. Will create a video soon on the specific use case that you have mentioned.. I mean by hosting on docker and chatting with docs. Thanks for the comment
Interesting idea of integration. But i think that since you have no bugs in this process, it must be automated. Also, ollama is nothing more than click-to-run tool to download and deploy different sorts of AI models, so in fact you don't really need it to run Mistral with PrivateGPT. Or am i wrong?
i am facing thiis error conda : The term 'conda' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:1 char:1 + conda create -n private1 python=3.11 + ~~~~~ + CategoryInfo : ObjectNotFound: (conda:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException
I am getting the sqlite3 error of using a unspported version, even the link specified does not solve my error can, I get the error when I run the ingest.py file
Hello, I am trying to follow your youtube video about "Private Chat with your Documents with Ollama and PrivateGPT" but when I type in my query after running the 'python privateGPT.py' part. I get the following error message "ValueError: Ollama call failed with status code 404. Details: model '7560' not found, try pulling it first". I'm not sure what this is talking about as I do the 'Ollama pull mistral' command before running the rest of the code. Any idea how to fix this?
Are you using the code from private gpt’s primordial version? What a great work! By the way, now they officially support integration with ollama. You probably wanna make a video about how to set this up.
If my company has a pre-existing Data Dictionary, is there a way to allow Ollama to integrate it so that it better understands the data I am working with?
Thank you for the amazing video! a quick question, are these local llms safe to use with private data? I want to use it for work but idk if the info will be shared with meta.
Hi, thanks for the tutorial. I am able to run the model using Ollama on windows but after everything the project runs and Asks for the Enter a query and if I add the questions it stucks there. In problems I am getting bekow issues such as Import "langchain.chains" could not be resolvedPylance Import "langchain.embeddings" could not be resolvedPylance Import "langchain.callbacks.streaming_stdout" could not be resolvedPylance etc Can you please help me out in understanding this issue and resolving it. Thanks.
yes. a good prompt. in the begining of the prompt mention that you are a chatbot and answer based on these documents only.... some modifications. but this works for openai api. i have tested in my current project that I am doing on some RAG applications.
Great video, took me a few tries getting Ubuntu configured (using wsl under windows) and updated before it would build the scripts, but I am very impressed. Thanks for all the effort you put into these videos 😁
Thanks for the great effort. I just have a question. When a file is ingested, then i want to ingest different file, should i delete the db folder first? so not to mix older ingested files with the current one i want to ingest. Also, after ingesting one of my csv files, i asked few questions. Then the answers i was getting were far off and was actually from another source, which i didn't provide, it was from a pdf book about getting rich or something. When i only ingested a csv file with numbers!
I got an error when I typed ollama run mistral. Got a a message saying that ollama command not found. I get this from within the terminal of Visual studio code. But I can run Ollama from the mac's terminal window no problem. Did I forget to do something? Thank you!!!
I've tried everything but it doesn't work for me...:-( ImportError: `PyMuPDF` package not found, please install it with `pip install pymupdf` I tried to install PyMuPDF but nothing changes
Just copied this from google, try this. "For the developers who are facing this issue on macOS, you need to install pip install PyMuPDF==1.20. 0 as PaddleOCR requires PyMuPDF
note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for PyMuPDF Running setup.py clean for PyMuPDF Failed to build PyMuPDF ERROR: Could not build wheels for PyMuPDF, which is required to install pyproject.toml-based projects @@PromptEngineer48
can I use this to extract pdf information from 100 pdfs? I need the same information extracted from each pdf and store it in rows, so I need a table of 100x (items to extract), i.e. extract Total Assets from a Balance sheet (will be my 1st column) and I need to do this task for 100 companies. can i use this code to do that? I feel that its more like a many to one thing this one, rather than many to many. Thanks so much for your content!
great video! it is very cool. I noticed that you implemented the Ollama on your mac, but can you update this to install in a server or the cloud that can provide this service to your friends? that would be helpful for your friends
Fantastic! Unfortunately it doesnt work :( When doing python ingest,py we get Your system has an unsupported version of sqlite3. Chroma requirwes sglite3 >= 3.35.0
@@PromptEngineer48 is there a way to correct it. I would really like to try your version. (I think I made it worse because after that I sudo apt get sqlite!)
Hi, I did what you said with some hesitation but it worked as you said . This is one good program to use to have a good look at a documents content and meaning. Thanks. BTW is there way to let the program give a full response with out telling it to continue with its explanation?
@promtEngineer, can you host it with basic UI interface, like people could upload their docs and ask questions, without doing all of this hardwork🙈, and also dont keep anyones personal documents with you
![Noob To Max With DRAGON REWORK In Blox Fruits [FULL MOVIE]](http://i.ytimg.com/vi/LnBMOinoOvA/mqdefault.jpg)
Dude … This is AMAZING! I was just looking for pushes in the right direction, but this actually does exactly what I was attempting to do! Thank you!
Glad I could help!
You've saved me from going down my RAG rabbit hole. The code is extremely detailed, clean, and easy to understand too. God bless.
Thanks for tuning in.
I've been looking for this for months. Thank you alot, dude! Subscribed :)
Thanks for the sub!
Just found out your channel and subbed after watching this video.
Very good quality video! Keep up the great content creation!
I am impressed with your motivation to answer the questions from your comment section.
Not every youtube channel cares about answering subscriber questions and doing content around what people comment , so very good job!
Thank you. It's my pleasure to be talking with my viewers..
i use "pip install tqdm ",The system prompts me: ModuleNotFoundError: No module named 'tqdm'
Make sure you have installed all the frameworks listed in the requirements.txt file. Rewind the video, he explains how to do it all at once. pip install -r requirements.txt
If you have done that, make sure you are working in the correct environment. The same environment you installed the python frameworks in. He explains that as well.
I get the same error. Everything installed successfully, and I'm in the same environment ... :0)
same here installed all the required frameworks but still got the error , what solved it
Can you still use the ollama API to search your documents? or does it need to be via the console?
API can do that
this is really high quality content, especially given the effort made in editing. the subtitle is a nice addition also.
Thank You for noticing the efforts... 😍 -- With Love (Prompt Engineer)
However, more than the subtitles, I want the main content to be more engaging.
I will like to say that this works just as well even today. Much thanks to the prompt engineer, you have made my life and others a lot easier.
Thanks for watching
Would like to see ollama with openai assistant or similar that can use a file and have assistants do action to the file such as execute code or clean up data etc.
Got it. Let me do some research.
If I give ollama pull mistral
I'm getting error that says could not connect to ollama app, is it running ?
Please give me a solution for it
simple, before typing ollama pull mistral, write ollama serve..
Actually using it in linux platform but still I gave the command ollama serve, I'm getting no GPU detected, Please give solution for this !
@@PromptEngineer48
Thanks!
Thank You so much !!
Followed your instructions here and installed on Raspberry Pi4. Works but of course painfully slow and chip approach 145 degrees F which slows things down as well. But it works and may try on Pi5. Was using a pdf manual for Viking drill press for document. Have to try something with just text.
Hmm. I see.
How do I remove the embedded Think and grow rich pdf file. I say this because when I add other file the query goes back to this pdf file and quotes it
delete the db and cache folders
Thank you, forgive me for sounding slow. You do mean remove or delete the chroma.sqlite3 file?@@PromptEngineer48
If I delete the cache folder name db , will this effect how the program will ingest the files I supply or will create a new db and then ingest the new files?
what is the system configuration that you are using and what is the response time for the query
Just a basic 8GB M1
About the context window. I have noticed that it cannot exceed over 2k tokens even though Mistral can support up to 8k. From what I have tested so for, it is like the bot identify itself as a GPT-3, is it because of the openai library?
yes. That is because we start everything compatible with OpenAI API. then we shift to opensource APIs. We could instead work for OpenSource APIs for the start. 😁
@@PromptEngineer48 So for this Ollama, the context windows will not be limited with 2k only right? It will be scale based on the model capability.
Yes.
@@PromptEngineer48 I wished that I could test it right now. Windows user need to wait.
We can try LocalAI. I will come up with a video
My brain is exploding because I was just using your code repository earlier today and then closed everything out. I'm trying to run it again and I'm now getting a bunch of depreciation LangChain warnings. I still get a response but the response is no longer referencing or citing the documents I placed in the folder. Any idea why this would happen?
my bad. i should have freezed the dependencies.
Fear not. I will create another video with fresh codes.
This is great, can you add or show how we could add gpu integration even better if it is over a GUI with privategpt 2.0?
Alright
Great video!! However if i follow the instructions my results are different. I created the source_documents folder an put in another pdf-file. When i then execute the "python3 ingest.py" the ingestion seems to work fine. But when i afterwards exceute the privateGPT.py and start to interact with the llm it still responses to "Think and Grow Rich"-Book.
Delete the db and cache folder
@@PromptEngineer48 can we not append the pdf files, do we have to keep removing them. when a new file is added and ingested, it should add the document into its cache and should be able to response from both the document isnt it
@@arvindelayappan3266 did you ever figure this out?
Thanks for doing this: It seems like when one clones the repository, you need to delete everything in the db folder or it will mess up the results of the information you are trying input yourself.
Awesome effort!! Your code worked right off the bat. Thanks for saving me a LOT of time.
Thanks
how can i get a good looking ui like a chatbot ? not on the command prompt like ui, instead , would like to see a web based ui with a send button
u can use streamlit, gradio, ollama web ui and so many other options
Does anyone get this warning - UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown (and subsequent crash)
I didnt get such
Thank you very much, all the videos you have made are great! However, when I was building the local llm, I found that there are multiple frameworks, such as privateGPT, localGPT, langchain, etc. Similarly, I found that there are very many choices for the llm as well as the vector database (e.g., hugging face vs. ollama), which gave me a big headache, and I was wondering if you could make a I was wondering if you could make a video that explains your recommendations for each part of the process of building a rag-based personal local document chat llm?
That would be a very good video but very less relevant as everyday we have so many updates. But this is something I can create. thanks for the idea. Definelty will work on that
@@PromptEngineer48 Thank you for your response! It's true that, as you say, the various programs are moving so fast. Since I'm hoping to be able to build something myself via langchain, I'm starting to work based on your github project, and some of the other out-of-the-box projects (e.g. open-webui, privateGPT, etc.) are just too heavy for me to get into and modify.
Cool.
It would be great to feed local AI with project code or framework, and be able to ask it questions about the code.
nice idea. i will try to implement the same.
Hi I am connecting from another computer. I have my Ollama in AWS cloud. How do I make it where I can train it like what you did here?
there was no training just injest and spit.. if you have ollama in AWS cloud, you need to somehow use that via api calls.
Great Tutorial, Why is it saying loading 235 new documents when it has only one in the source_documents folder. I had 2 in mine and it said 8, wondering why.
It's because of the chunking.. I had put only one document but it was chunked into many pieces.
hi, can you update this so we can use LLama 3.1 instead, please provide a version that works with Apple silicon as this one fails on my M3 Mac
thank you, sorry but i don't see the old chat interface :-) i have 2 questions
how can i remove a ingested document that i don't need?
ist it possible to chat with the docker interface ? or is it only for terminal??
1. U can remove the file. Then delete the db folder and __pycache__ folder.. then run python ingest.py again and python privateGPT.py again.
2. Web UI interface is not integrated here right now. But that is some ok my pipe line as well. I am working on that.
Do you have the pdf document you tried? Would like to try it myself
It's the think and grow rich book. Just search for the book on the internet
This work very well but it has issues ingesting docx, pptx and ods files without a pip python install
Thank you for this great content! I was able to get this working on my M1 Mac. I was able to run `python ingest.py` and the `python privtagpt.py` commands. However, when I asked it to summarize the document I had uploaded, it referenced the "Think and Grow Rich" document that you showed in the video, rather than the test document I uploaded to the source_documents folder. How do I clear out the embeddings from the "Think and Grow Rich" document, or clear the chroma db database embeddings completely before running the ingest.py command again?
Delete the db and cache folder
@@PromptEngineer48 Thank you! Got it to work exactly as expected! Thank you for all the great content!
Welcome
This is awesome, but still not bale to chnage the base url for ollama, is there any way i can change that?
ModuleNotFoundError: No module named 'tqdm'
even after installing the library properly. Any fix ?
Were you about to solve?
@@PromptEngineer48 yeah ..updating the package fixed it for me. Thanks again for this awesome tutorial
@@varun_tech7You solved the problem yourself. Congrats.
@@varun_tech7 which package did you update? thx
@@raminderpalsingh123 I don't exactly remember, just make sure all package is up to date
Hi, while setting up Virtual environment I'm getting an error, 7:39 what programs do I need? I have installed Miniconda and it's still no running the line. Thks
then try with Ananconda
Wonderful, thank you! My dream would be to feed my local model with all my books in epub or pdf format just once and the model will keep a memory of those. Then from there I have so many ideas but not sure if feasible, maybe someone can help? 1) for each book create a mind map of concepts 2) a diagram of how each book is related to the others (citations, same authors, same topic, related concepts) 3) given a question or a topic the system can point me to which book is better to read. Probably impossible at the moment, isn't it?
Wonderful idea. I will dedicate time for a POC
@@PromptEngineer48 Awesome! Another one could be to automatically find the book category. Since the system has already ingested all the books it already knows the discussed topics in each book and from there it can assign each to an ontology of category and subcategories. Like non-fiction/self-help, fiction/novel, non-fiction/self-help/personal growth, and so on. I have other 100 of these ideas, ping me if you need more :D
I would also be interested in something like this if possible as it would be amazing to do this in a repository of books that you know. If you could do this with a non-trivial number of books, say, 100-150 (from Project Gutenberg), it would be a great application of local LLMs.
how can we get chatGPT alike local LLm, i want to use it for Research and forecasting and predictions
there are research going on. you need to find it from huggingface or ollama .. see the benchmarks and decide which model you want to use. since we have so many options, I cannot name one. but it's a case to case basis.
Keep getting a error when doing python3 ingest.py says line 8 no module named langchain
pip install langchain langchain_community
Can you tell, how to use other models like llama3, without using mistral?
Hi, can you create a video on how to clean our own data which is in my CSV file which is best to answer our query very effectively?
Yes I can. but to be more clear.. you want to use a data cleaner llm which will give clean you csv file?
@@PromptEngineer48 Okay pls go ahead
Well explained. Can you suggest if we want to pass a onenote how we can pass it.
I am getting answers from out of pdf also.How to restrict that to only pdf specific?
Ingest only the pdf.
@@PromptEngineer48 even after ingesting the PDFs,I am getting answers for some questions that are not in pdf.Is it because of the already trained model?
Could you provide suggestions on how to enhance the conversational capabilities of this bot?
Prompts and proper chains ⛓️
Are we using poetry here or not? because that part was not there in the video.
Yes.
Is the full code available on the GitHub repository or are their some scripts missing using git ignore?
No. Everything in GitHub repo
I'm getting CondaError: Run 'conda init' before 'conda activate' during my installation. I did try conda init but then it says "no action taken". As usual I can't get a step-by-step tutorial to work.
So you were able to create a conda environment?? Using conda create -n your-name python=3.11??
Yes. Followed everything up until the activation part
Although I don't get the (base) in front of my path like you did after
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Where the knowledge base is stored? It keeps track of all pdfs already processed?
The are two folders created automatically names db and cache.
Hello, i keep getting error ERROR: Could not find a version that satisfies the requirement onnxruntime>=1.14.1 (from chromadb) (from versions: none)
ERROR: No matching distribution found for onnxruntime>=1.14.1
I had the error for ChromaDB too. If you Google for them you will probably find an answer. For ChromaDB this solved it for me:
- sudo apt install python3-dev
- sudo apt-get install build-essential -y
Nice video, very informative! What do you use to generate subtitles? thanks
Thanks.. Capcut
Thanks for your tutorial. I am trying this on Windows pc. I am in anaconda prompt. I am stuck at this error "Error: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based project". I tried installing installing visual studio build tools but the error still persists. Any idea?
I believe we need .NET SDK, .NET Framework and other tools using the build installer
@@PromptEngineer48 all installed but still wont run.
got the same error, did you manage it to run?
I can not update the db, when I ask the agent about the new added document, it still gives answers about the document in this video. It is kind of confusing.
Delete the db and cache folder.. then upload your own document.
Thank you, you are right, and really helpful!@@PromptEngineer48
I couldn’t get Conda to work after installing Conda . The installed files are in /opt/miniconda3 but whenever I run Conda, it’s says Command Not Found
www.anaconda.com/download
You need to install anaconda. however, you could use the .venv instead of conda. We just need a virtual environment.
@@PromptEngineer48ok managed to get Conda working. However when I run the python3 ingest.py, I got an error: line6, in from tqdm import tqdm. ModuleNotFoundError: No module named ‘tqdm’
Ok made more progress but now stuck in pymupd. Tried installing it but keep getting message Requirement already satisfied: /opt/anaconda3/envs/privategpt/lib/python3.11/site-packages
pip install tqdm should have worked
Thanks, that’s a great video! I’m testing privategpt for some time and I would love to know if you’re experiencing hallucinations from the chat? And yes mistral seems to be a good model but Zephyr and Dolphin seems to give better answers with a little less performance, depending on the context. Therefore, I couldn’t Get rid of some hallucinations, I would say the reliability of information is 45-65%
Yes. Hallucinations are there. Need a good model in future
Hey! Are you able to show how to use auto-mem-local using Ollama? Thanks!
Noted !!!
Hello @PromptEngineer48 , I new to python and want to learn this. I followed your instruction in your great video, but had many errors when installing modules in requirements.txt. Any update?
why dont i come up with an updated video.. please give me like a week or so.
@@PromptEngineer48 very nice... thank you sooo much....
Is there a limitation on the max number of personal documents that I can upload to PrivateGPT?
No. but then the vectorstore gets confused and not able to get the relevant chunks.
Hi, Ollama is running, I input query but nothing receive answer, termial is blank. What should I do?
Try with different LLMs. a smaller version please
Very interesting, easy to follow. I tested this with a music instrument manual, and it wasn't giving accurate answers at all. Any ideas on how to improve this? It's unusable for this type of document. Makes me wonder how accurate it would be with any content. Thank you!
That was pretty old stuff. Please watch the recent videos on my channel.
This has been very instructive, thanks!
Is there an LLM that's better than Mistral at working with very technical documents, i.e. lengthy API implementation documents? I set up my environment exactly as instructed, got my docs indexes and could get some answers on them. But things get murky when I go very specific, with incorrect or incomplete answers.
Try with mixtral 7x8b
Great tutorial, one of the best on the web!! Thanks for your time and effort! Upvoted 👍
Thanks so much.
Awesome idea. Now use Streamlit or Flask as GUI and the solution is perfect. Thanks for the inspiration. Questions: 1. which OS are you using? 2. python version? 3. do you rather use CPU or GPU? Would an M3 with 128 GB also be sufficient for a quick training / fine tuning of hugging face models up to 20B? Thanks for the answer in advance.
Hp
1. Mac OS. Pretty basic. Macbook Air M2, 8 GB
2. python version 3.9 ++ version
3. M3 with 128 GB.. Hmm when you say fine tuning it depends on the model. But here is a rough calculation. If you have a 20B parameter model in 32 bits, then you need 20x32/8 = 80 GB GPU. your system should be able to do the fine tuning. Else please go for 4 bit quantization, then the requirement will reduce by 8 times, now requiring only 80/8 = 10 GB of GPU.
I'm using streamlit for a great little interface.
How would you do this if you have ollama in docker? Or even cloud gpu like runpod? Or Linode? Seems like a good video for the future as you explain,and teach better than a majority of the creators.
Okay. Will create a video soon on the specific use case that you have mentioned.. I mean by hosting on docker and chatting with docs. Thanks for the comment
same here, I'm using dockerized ollama.. would be great to have privategpt with dockerized ollama..
Okay. Got the requirements
Now it's my turn to create that. 😄
can you provide a similar use case setup for windows as well, now that Ollama is available on windows
No difference. Just use the code on windows. Make sure to install Ollama on windows.
Interesting idea of integration. But i think that since you have no bugs in this process, it must be automated. Also, ollama is nothing more than click-to-run tool to download and deploy different sorts of AI models, so in fact you don't really need it to run Mistral with PrivateGPT. Or am i wrong?
Now with more interesting integrations, we can scrape privateGPT itself and use Ollama to code up our projects natively. Yes you are right
i am facing thiis error
conda : The term 'conda' is not recognized as the name of a
cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the
path is correct and try again.
At line:1 char:1
+ conda create -n private1 python=3.11
+ ~~~~~
+ CategoryInfo : ObjectNotFound: (conda:String) [],
CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
docs.anaconda.com/free/anaconda/install/windows/
install the anaconda
how is this approach different from RAG, can you elaborate please?
It is RAG
@@PromptEngineer48 but i'm confused you replied to the message just below for his question which assumes it is not RAG?
I am sorry for any confusion. It is a RAG system. any use input of pdfs etc will be used to reply to the user's questions.
I want to try to give it some ability to troubleshoot for me, like conduct pings and traceroutes.
looks good
I am getting the sqlite3 error of using a unspported version, even the link specified does not solve my error can, I get the error when I run the ingest.py file
Were u able to solve?
Hello, I am trying to follow your youtube video about "Private Chat with your Documents with Ollama and PrivateGPT" but when I type in my query after running the 'python privateGPT.py' part. I get the following error message "ValueError: Ollama call failed with status code 404. Details: model '7560' not found, try pulling it first". I'm not sure what this is talking about as I do the 'Ollama pull mistral' command before running the rest of the code. Any idea how to fix this?
try this ollama run mistral:latest
@@PromptEngineer48 It let's me run mistral:latest but how will this fix my error above? Do I have to add that to the python code somewhere?
Are you using the code from private gpt’s primordial version? What a great work! By the way, now they officially support integration with ollama. You probably wanna make a video about how to set this up.
If my company has a pre-existing Data Dictionary, is there a way to allow Ollama to integrate it so that it better understands the data I am working with?
yes. that could be included technically.
Thank you for the amazing video! a quick question, are these local llms safe to use with private data? I want to use it for work but idk if the info will be shared with meta.
U can switch off internet. Safe 🔐 or not, we cannot guarantee, it may so happen that once u connect to internet the data may be transferred.
@@PromptEngineer48 okay, so the advantage of running an llm locally is just to save some $$ from API calls?
Hi, thanks for the tutorial. I am able to run the model using Ollama on windows but after everything the project runs and Asks for the Enter a query and if I add the questions it stucks there. In problems I am getting bekow issues such as
Import "langchain.chains" could not be resolvedPylance
Import "langchain.embeddings" could not be resolvedPylance
Import "langchain.callbacks.streaming_stdout" could not be resolvedPylance
etc
Can you please help me out in understanding this issue and resolving it.
Thanks.
There are bugs in windows.. you can try with Linux on windows.
is there the posibility to take responses only from your documents?
yes. a good prompt. in the begining of the prompt mention that you are a chatbot and answer based on these documents only.... some modifications. but this works for openai api. i have tested in my current project that I am doing on some RAG applications.
Great video, took me a few tries getting Ubuntu configured (using wsl under windows) and updated before it would build the scripts, but I am very impressed. Thanks for all the effort you put into these videos 😁
🤗 welcome. Trying to bring the best.
what is "wsl under windows?" I'm running Linux. No Windows here. His instructions in the video don't work for my system at all.
line 6, in
from tqdm import tqdm
ModuleNotFoundError: No module named 'tqdm'
gets stuck at this
pip install tqdm should solve the issue.
can you help me? after the python ingest.py i receive a error.. :-(
please mention the error !!
i try to copy here the error but it does't accept @@PromptEngineer48
i have a new Mac with chip M2
the problem is the hnswlib library
in Mac you should do python3 instead of python.. so please type python3 ingest.py
Thanks for the great effort.
I just have a question. When a file is ingested, then i want to ingest different file, should i delete the db folder first? so not to mix older ingested files with the current one i want to ingest.
Also, after ingesting one of my csv files, i asked few questions. Then the answers i was getting were far off and was actually from another source, which i didn't provide, it was from a pdf book about getting rich or something. When i only ingested a csv file with numbers!
Yes u need to delete the db file first
How does it knows it should use mistral if I have several models downloaded?
that is hardcoded
@@PromptEngineer48 actually not. This written in config. Take a look
do you know how to stop Ollama afterwards? it continues to run in the background even after trying to end the process multiple times.
An icon on the top.. appears. Press and quit
Will it recalculate embeddings everytime I add more pdf documents?
yes
@@PromptEngineer48 I mean it should only compute embeddings for the new documents only, not for entire set.
Great video! Can I connect Langchain with it? Or is it included. Im a newbie on it :D
Yes u cN
I got an error when I typed ollama run mistral. Got a a message saying that ollama command not found. I get this from within the terminal of Visual studio code. But I can run Ollama from the mac's terminal window no problem. Did I forget to do something? Thank you!!!
try to change the type of terminal that you are using. i mean zsh or command prompt.
Did not work. Errors. If execute Python ingest.
I've tried everything but it doesn't work for me...:-(
ImportError: `PyMuPDF` package not found, please install it with `pip install pymupdf`
I tried to install PyMuPDF but nothing changes
Just copied this from google, try this.
"For the developers who are facing this issue on macOS, you need to install pip install PyMuPDF==1.20. 0 as PaddleOCR requires PyMuPDF
thanks, i try it now @@PromptEngineer48
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for PyMuPDF
Running setup.py clean for PyMuPDF
Failed to build PyMuPDF
ERROR: Could not build wheels for PyMuPDF, which is required to install pyproject.toml-based projects
@@PromptEngineer48
on my mac it ist not possible to install a old PyMuPDF 😞@@PromptEngineer48
😢
Can private GPT be run for a web type interface similar to your previous video ?
Is there anything which is not possible. 😅😀
can I use this to extract pdf information from 100 pdfs? I need the same information extracted from each pdf and store it in rows, so I need a table of 100x (items to extract), i.e. extract Total Assets from a Balance sheet (will be my 1st column) and I need to do this task for 100 companies. can i use this code to do that? I feel that its more like a many to one thing this one, rather than many to many. Thanks so much for your content!
If I understand that correctly, that could be hard coded. I think we don't need an LLM here
great video! it is very cool. I noticed that you implemented the Ollama on your mac, but can you update this to install in a server or the cloud that can provide this service to your friends? that would be helpful for your friends
Yes you can! I will try to bring in a video
Good work!
I found your video well done!
great video. now can u do it in streamlit so there's user interface to chat and how can we host it online? like render?
Okay. Streamlit and render integration got it. Will do that.
How do you turn sources off? Just get the answer?
python privateGPT.py --hide-source
Try this
hi, ask a question. How can I change the model for example to llama3 with model 8b-instruct-q6_K?? please
go to line 12 of the privateGPT.py file and change the mistral to whatever model your heart desire.
which is the best vision model to extract entities from complex invoices ?
If you are talking about open source, then I have to go with ollama.com/library/llava
If closed source, then openai
The response speed is slow on MacBook Air
okay. Try switching the model
can you make a video for the front end as well?
Front end. okay. which interface would you like.. like gradio, streamlit etc. any specific requirements. my default would be gradio. would that work?
Yes, Gradio will be suitable! Thank you for your quick response. I appreciate your efforts. I look forward to watching your video
Okay. On it
Thank you very much t @PromptEngineer48.Looking forward to it.
Fantastic! Unfortunately it doesnt work :( When doing python ingest,py we get Your system has an unsupported version of sqlite3. Chroma requirwes sglite3 >= 3.35.0
My bad.i have committed a sin by not freezing the library versions
@@PromptEngineer48 is there a way to correct it. I would really like to try your version. (I think I made it worse because after that I sudo apt get sqlite!)
I will have to search my old files.
Is similar solution possible using 'LM Studio' for users having Windows machines?
Not right now. But if u follow the discord of lmstudio. They have plans
@@PromptEngineer48 i installed wsl in my windows machine and followed your tutorial, everyhting worked fine. Great tutorial btw, thank you :)
is there a way to view the actual embeddings value from ChromaDB?
Sure it is
@@PromptEngineer48 How exactly can I view them ?
It Is possibile to load legacy code (i.e vb6 )and database schema and query how this code does?
I should be possible
Hi, I did what you said with some hesitation but it worked as you said . This is one good program to use to have a good look at a documents content and meaning. Thanks. BTW is there way to let the program give a full response with out telling it to continue with its explanation?
I think there should be a verbose flag, which you can set to False.
@promtEngineer, can you host it with basic UI interface, like people could upload their docs and ask questions, without doing all of this hardwork🙈, and also dont keep anyones personal documents with you
Ok
You're a life SAVER... many thanks
Welcome
@@PromptEngineer48 can i apply this inside streamlit web app so that users ask and get the answer on ui? Can these models be deployed ?