Hi Everyone, I uploaded a new PrivateGPT v 0.4.0 build Video, This Video shows how to build PrivateGPT on Windows with Ollama support. Check it out on the link below. 🔗PrivateGPT 0.4.0 with Ollama Support. ruclips.net/video/Ww2oelybDz8/видео.html 🔗
I've installed both 2.0 version and the 4.0 version and both works. But even though the ollama version is really very practical compared to this one, the upload of the files are taking tooo much time around 15-20 times longer for each file. With version 2.0 is also not fast, but at least the cpu load shows around 80% almost full load and the parsing of the embeddings are taking for example for 600 pages pdf document around 10-12 min whereas with 4.0 (ollama) 2 hours only for one file. A huge 2700 page Book could not even be finished even after 6 hours. The reason seems to be that the cpu has almost no load during the upload 5% => of courese it will take 16 times more time if consider the load propotional with parsing. With 4.0 version something is wrong. I am wondering whether the failure is in my system or not, if someone could share here his experience regarding the upload time and load, this would help me understand where the failure could be
If you get this error when using this command poetry install --with ui,local Group(s) not found: local (via --with), ui (via --with) Use this command poetry install --extras "ui vector-stores-qdrant llms-ollama embeddings-huggingface" You're Welcome :)
As i installed the poetry install --extras "ui vector-stores... it installed only 12 packages, in the video he got a lot more than that. Now when I run the poetry run python-m private_gpt (in the video 7:15) it shows me that I am missing --extras llms-llama-cpp. after I got this installed, something different went wrong and is missing and it is in this circle that I am downloading something and something other is missing. Is there any way to get this running?
to anyone who gets hit with this error: ImportError: Local dependencies not found, install with `poetry install --extras llms-llama-cpp` you need to continue adding the dependency errors it throws at you into the install command. since you cant install llms-llama-cpp, here is what you would use poetry install --extras "llms-llama-cpp ui" and continue adding any more it throws at you. for example, the next error i got said ImportError: Local dependencies not found, install with `poetry install --extras embeddings-huggingface` so my next command was poetry install --extras "llms-llama-cpp ui embeddings-huggingface" this kept going until the command that got it working was poetry install --extras "llms-llama-cpp ui vector-stores-qdrant embeddings-huggingface" try to just keep adding the dependency names to the install command until it works
YOOO, IT WORKS: poetry install --extras "llms-llama-cpp ui vector-stores-qdrant embeddings-huggingface" Thank you, in the cmd it says: uvicorn.error - Uvicorn running on 0.0.0.0:8001 (Press CTRL+C to quit) just change it to 127.0.0.1:8001
This might help as well, this resolved my issue. set PGPT_PROFILES=ollama poetry install --extras "ui llms-sagemaker embeddings-sagemaker vector-stores-qdrant" poetry run python -m uvicorn private_gpt.main:app --reload --port 8001
Impressive. I loaded the instruction manual for my HAAS VF4SS mill (24meg PDF) and asked multiple questions about g-code. It gave correct answers every time. I input some code and asked for a description of what the machine would do if I ran that code. Very detailed and correct answer. I am impressed. Thank you. P.S. I'm running the Mistral LLM and it's only taking up about 7gig of VRAM and is pretty fast with the answers.
Glad to hear, thanks for the feedback. There is a new video that offloads the LLM and GPU's to Ollama. Have a look on the channel if you want to check out the new version. Thanks for reaching out. Updated video PrivateGPT. ruclips.net/video/Ww2oelybDz8/видео.html
I built this up for both the recommended ollama and llama. Ollama didnt seem to utilize much if any of the GPU and documents worked but were too slow to be useful. Llamma works much better. I am sure the ollama issues will be resolved and long term may turn out to be the better method but as of today the llama profile works better
Hi, thanks for the feedback! I dont know if you checked out the new version running on Ollama as backend. Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
Done, Thanks! Link to the new video below.... I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
Video is still valid up until 6:07. A lot of what comes after that has changed in the project. You are going to have to go to the project docs and follow the instructions there for the remainder of the install. You now have more options and it branches after the 6:07 time stamp. If you did everything correctly up until this time stamp and then follow the recommended local,ollama-powered setup you should be fine. When I did this on windows the ollama serve command failed due to an error. You can skip that command and just open the ollama application and proceed with the next step. Note: windows you probably need to break the PGPT_Profiles=ollama make run into two lines. Breakpoint after ollama
Hi, thanks for the feedback. I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
I watched it and it is a good video. I appreciate the work you put into your content. How was your file ingestion with ollama? I couldn't get good performance and had to switch to llama to get it back to reasonable speed.@@stuffaboutstuff4045
SUCCESS! I managed to get the server running by following the video until 6:07 at which point (as noted in other comments), instead of *poetry install --with ui,local* I used: *poetry install --extras "llms-llama-cpp ui vector-stores-qdrant embeddings-huggingface"* After that, carrying on from 9:58 (running locally with an nvidia gpu) I had an error: "The CUDA Toolkit v12.4 directory '' does not exist. Please verify the CUDA Toolkit is installed properly or define the CudaToolkitDir property to resolve this error." To fix that I needed to use regedit to add a new key called *InstallDir* under *computer\HKEY_LOCAL_MACHINE\SOFTWARE\NVIDIA Corporation\GPU Computing Toolkit\CUDA\v12.4* (or whichever is your cuda toolkit version) and set the value to *C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4* (this location is your cuda_path which can be found in environment variables within control panel).
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html Thanks for the feedback above!
Thanks for the feedback! If you are interested updated build video released today. I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
I I tried it last week and it worked great. Now i got a graphics card and wanted to reinstall it. Unfortunately I keep getting an error that I haven't been able to fix for hours.: (privategpt) PS C:\pgpt\privateGPT> poetry install --with ui,local Group(s) not found: local (via --with), ui (via --with) what am I doing wrong?
I'm trying to get this installed too as I type, but I believe the instuctions on this video are already outdated. It looks like the developer changed the syntax and added more options. I got past this point in the instructions with the following command based on the installation instructions on the privateGPT documentation website: poetry install --extras “ui llms-ollama embeddings-ollama vector-stores-qdrant” It then installed numerous libraries similar to the video, but then gave me an error on the next step, but the documentation website says I need to set an environment variable, install Ollama, and download a model, which I haven't done yet, so that is probably the cause of my error.
How solve: (privategpt) C:\pgpt\privateGPT>poetry install --with ui,local Group(s) not found: local (via --with), ui (via --with) (privategpt) C:\pgpt\privateGPT> I did all the steps to "(privategpt) C:\pgpt\privateGPT>poetry install --with ui,local" with no error
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
Hi, Have you had a look at the updated video I posted. ruclips.net/video/Ww2oelybDz8/видео.html You can have a look at the ingestion possibilities here. docs.privategpt.dev/manual/document-management/ingestion
Pleasure! Updated version should you want to check it out... I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
please help ! 6:33 , following everything step by step, but after >poetry install --with ui,local it says Group(s) not found: local (via --with), ui (via --with)
I changed "poetry install --with ui,local" to "poetry install --extras "ui vector-stores-qdrant llms-ollama embeddings-huggingface"" (found on page - Private GPT Docs) and it started installing, hope the rest will go smoothly
Hi, did you perhaps check out the updated PrivateGPT build on the channel. Let me know if the below video helps get you up and running. ruclips.net/video/Ww2oelybDz8/видео.htmlsi=6wj-QoaPAfsZjMiM
Hi, thanks for reaching out. Please have a look at the updated build video on the channel. You can have a look at the link below. ruclips.net/video/Ww2oelybDz8/видео.html
I got an error after: poetry run python -m private_gpt it stopped generating, did not give me the website and I got this: ImportError: Local dependencies not found, install with `poetry install --extras llms-llama-cpp` is there any chance to get it working?
File "C:\pgpt\privateGPT\private_gpt\components\llm\llm_component.py", line 37, in __init__ raise ImportError( ImportError: Local dependencies not found, install with `poetry install --extras llms-llama-cpp` >whole error
@@sibeejeyachandran8174 @danda3688 you need to continue adding the dependency errors it throws at you into the install command. since you cant install llms-llama-cpp, here is what you would use poetry install --extras "llms-llama-cpp ui" and continue adding any more it throws at you. for example, the next error i got said ImportError: Local dependencies not found, install with `poetry install --extras embeddings-huggingface` so my next command was poetry install --extras "llms-llama-cpp ui embeddings-huggingface" this kept going until the command that got it working was poetry install --extras "llms-llama-cpp ui vector-stores-qdrant embeddings-huggingface" try to just keep adding the dependency names to the install command until it works
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
Thank you so much! It took me a couple of tries, but following your instructions carefully proved successful! I just ran into an issue with CUDA when running the command: $env:CMAKE_ARGS='-DLLAMA_CUBLAS=on'; poetry run pip install --force-reinstall --no-cache-dir llama-cpp-python The filename, directory name, or volume label syntax is incorrect.
7:21 I dealt with the previous error reading your comment, but this one... Installing what it says in cmd doesn't do anything. return callable(*full_args, **dependencies) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\pgpt\privateGPT\private_gpt\components\llm\llm_component.py", line 37, in __init__ raise ImportError( ImportError: Local dependencies not found, install with `poetry install --extras llms-llama-cpp`
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
It works thanks for the sharing. Just a question...is it possible to increase/force the use of the GPU... mine only uses aroung/no more than 20% of the GPU capacity?
Hi, Looks like there is a new build again with a lot more options. I will look at building it the weekend. If you want you can check the new options available. docs.privategpt.dev/installation/getting-started/main-concepts
@@stuffaboutstuff4045 I've reseted the PC and done a new build, and it's quite fast now, although still not all PC resources are being used (CPU and GPU). The only change made now was: Before: poetry install --with ui Now: poetry install --extras "ui llms-llama-cpp embeddings-huggingface vector-stores-qdrant"
File "C:\Users\INTEL\anaconda3\envs\privateGPT\Lib\site-packages\llama_index\llms\llama_cpp\base.py", line 104, in __init__ raise ValueError( ValueError: Provided model path does not exist. Please check the path or provide a model_url to download.
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
Hi, did you perhaps check out the updated PrivateGPT build on the channel. The newer edition you can completely offload the local processing. ruclips.net/video/Ww2oelybDz8/видео.htmlsi=6wj-QoaPAfsZjMiM
Improving performance using openai API key is not working on my PC. It does not give me answers when I try to ask questions. What do I do? Should I generate another API key?
Hi, It looks like there is a new build again with a lot more options. I will look at building it the weekend. If you want you can check the new options available. docs.privategpt.dev/installation/getting-started/main-concepts
Pipx not installing on Win10 (64bit). After I enter command " pip install pipx ", nothing happens and it reverts to " C:\WINDOWS\System32> " and awaits new commands.
Hi, difficult one it should not do nothing. I assume you are in a privileged mode command prompt? if you run pipx is it not maybe installed already. What happens when you run pipx ensurepath. Is your pip working? What happens when you try and upgrade pip does it return anything? python -m pip install --upgrade pip. Let me know and I will try and assist..
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
6:39 after typing (privategpt) C: \pgpt \privateGPT>poetry install --with ui, local I’m getting this error Group(5) not found: local (via --with), ui (via --with) what should I do
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
Following the tutorial to the letter, fails at: Building wheel for llama-cpp-python (pyproject.toml) ... / ERROR: Failed building wheel for llama-cpp-python Failed to build llama-cpp-python ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects
Hi, these build errors are usually due to missing required SW or privileges. To resolve privilege issues run all commands in Admin mode terminals. Make sure your Python is within 3.11.x This is important. Installed and added to path. Install VS community and make sure you selected Desktop Dev with C++ and Universal Windows platform development. As confirmed make sure all steps are follwed from 3:52 in the video. Let me know if any of the above resolved the issue..
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
Doesn't using openAI's model defeat the purpose of an offline 'Private' gpt session? Otherwise.. why would you need an API key? Maybe the authentication is safe once you verify but it's still a little sketchy to my ignorance.
Hi, I think at this stages it helps out when someone has technical constraints on CPU's and GPU's. Agreed the whole idea is to run offline but I think there are scenarios where using OpenAI comes in handy. Thanks for the feedback!
Hi, thanks for making contact. Yes it must be Python 3.11.x. I picked this up as well. Try using the version I have in the video description. Let me know if resolved...
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
I have installed PrivateGPT on my machine, it worked fine and I am able to ask questions as well.. once I closed cmd. I don't know how to again lunch PrivateGPT. You have any idea how to relaunch the PrivateGPT in cmd or uvicorn? Thanks in advance
Hi, thanks for reaching out. Please have a look at the updated build video on the channel. You can have a look at the link below. Yes, you have to use 3.11.xx. I think I make it more clear in the new video. ruclips.net/video/Ww2oelybDz8/видео.html
Hopefully someone can help with this. So I installed everything no errors and private gpt worked fine until I installed the Cuda stuff to get GPU support. Now I get these errors: When trying to do the 'conda activate privategpt' I get 'conda is not recognized as an internal or external command, operable program or batch file. Then when trying to run private gpt with 'poetry run python -m private_gpt' I get this: C:\JGPT\privateGPT>poetry run python -m private_gpt Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "C:\JGPT\privateGPT\private_gpt\__main__.py", line 3, in import uvicorn ModuleNotFoundError: No module named 'uvicorn' Any ideas? Thanks.
Hi, Make sure you run the command in a conda terminal and not a command prompt window. Otherwise conda is not in your windows path anymore. Let me know if resolved?
Hi, sounds like poetry might not be globally available. Check 3:47 into the video. Did you configure it globally by following the steps from there exactly? From the bit of upgrading pipx in a admin command prompt (Not Conda) up to running pipx ensurepath. Let me know if this shines light on your issue. Thanks for reaching out!
Hi, did you see this reply? Hi, sounds like poetry might not be globally available. Check 3:47 into the video. Did you configure it globally by following the steps from there exactly? From the bit of upgrading pipx in a admin command prompt (Not Conda) up to running pipx ensurepath. Let me know if this shines light on your issue. Thanks for reaching out! Please confirm if those were followed...? Let me know if resolved or if you still stuck with the error.
I am stuck on the GPU install step: $env:CMAKE_ARGS='-DLLAMA_CUBLAS=on'; poetry run pip install --force-reinstall --no-cache-dir llama-cpp-python The filename, directory name, or volume label syntax is incorrect. Any help is much appreciated
removed the CMAKE the rest ran but still very slow so I am unsure if it is running on my GPU. Any way to check. Thanks again for the amazing walkthrough!
Hi, You can open your task manager or resource manager on windows. Execute a search in PrivateGPT and see what takes the hit the CPU or GPU. One of them should spike and show you where the processing is being handled.
@@stuffaboutstuff4045i have tha same problem, i even changed it to localhost:8001 still dnt open 20:24:18.638 [INFO ] private_gpt.ui.ui - Mounting the gradio UI, at path=/ 20:24:18.689 [INFO ] uvicorn.error - Started server process [6736] 20:24:18.689 [INFO ] uvicorn.error - Waiting for application startup. 20:24:18.690 [INFO ] uvicorn.error - Application startup complete. 20:24:18.692 [INFO ] uvicorn.error - Uvicorn running on 0.0.0.0:8001 (Press CTRL+C to quit)
If you follow all the steps in the video up to 7:00 mark it should be configure for local LLM and CPU. Did you make sure all steps were followed up to that point? Incl setting it up as local and executing the scripts to download the LLM.
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0. ruclips.net/video/Ww2oelybDz8/видео.html
Thank you so much I appreciate. Hi this project is working on my pc but 1 thing is not missing which is "LLM chat" why it is not responding" the answer I have download the model: TheBloke/Llama-2-7B-Chat-GGUF llama-2-7b-chat.Q4_K_M.gguf but not working i have changed the name in setting.yaml and setting-local.yaml in it. llamacpp: llm_hf_repo_id: TheBloke/Llama-2-7B-Chat-GGUF llm_hf_model_file: llama-2-7b-chat.Q4_K_M.gguf? please tell me the solution or any video reference tutorial Thank you so much in advance.
Hi, just checking in to see if you were able to resolve this issue. I have released an updated video to build the latest version correctly. You can check out PrivateGPT v0.4.x using this link. Let me know if you are sorted and got it up and running correctly. Thanks for reaching out. ruclips.net/video/Ww2oelybDz8/видео.html
Hi, Thanks v4 was very very helpful for me. Now Please make the video on that we could find the specific text to draw the live rectangular bounding boxes on doc with accuracy where the text locate in page or line... Regex may also be use in that but please make video on it. Thank you.
Hi, Can you help me with this error? I have downloaded the tool kit and it is able to find it, But still it gives this error. Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.3/include (found version "12.3.107") -- cuBLAS found CMake Error at C:/Windows/System32/privateGPT/myenv/Lib/site-packages/cmake/data/share/cmake-3.28/Modules/CMakeDetermineCompilerId.cmake:529 (message): No CUDA toolset found.
It might be trying to use the VS Build tools. Can you check if this article is applicable to your problem. stackoverflow.com/questions/56636714/cuda-compile-problems-on-windows-cmake-error-no-cuda-toolset-found
Hi Everyone,
I uploaded a new PrivateGPT v 0.4.0 build Video, This Video shows how to build PrivateGPT on Windows with Ollama support. Check it out on the link below.
🔗PrivateGPT 0.4.0 with Ollama Support.
ruclips.net/video/Ww2oelybDz8/видео.html
🔗
I've installed both 2.0 version and the 4.0 version and both works. But even though the ollama version is really very practical compared to this one, the upload of the files are taking tooo much time around 15-20 times longer for each file. With version 2.0 is also not fast, but at least the cpu load shows around 80% almost full load and the parsing of the embeddings are taking for example for 600 pages pdf document around 10-12 min whereas with 4.0 (ollama) 2 hours only for one file. A huge 2700 page Book could not even be finished even after 6 hours. The reason seems to be that the cpu has almost no load during the upload 5% => of courese it will take 16 times more time if consider the load propotional with parsing. With 4.0 version something is wrong. I am wondering whether the failure is in my system or not, if someone could share here his experience regarding the upload time and load, this would help me understand where the failure could be
If you get this error when using this command poetry install --with ui,local
Group(s) not found: local (via --with), ui (via --with)
Use this command
poetry install --extras "ui vector-stores-qdrant llms-ollama embeddings-huggingface"
You're Welcome :)
Bro thank you!
As i installed the poetry install --extras "ui vector-stores... it installed only 12 packages, in the video he got a lot more than that. Now when I run the poetry run python-m private_gpt (in the video 7:15) it shows me that I am missing --extras llms-llama-cpp. after I got this installed, something different went wrong and is missing and it is in this circle that I am downloading something and something other is missing. Is there any way to get this running?
poetry install --extras "llms-llama-cpp ui vector-stores-qdrant embeddings-huggingface"
Said that to me, use command above @@danda3688
bro you're a life saver thank you
to anyone who gets hit with this error:
ImportError: Local dependencies not found, install with `poetry install --extras llms-llama-cpp`
you need to continue adding the dependency errors it throws at you into the install command. since you cant install llms-llama-cpp, here is what you would use
poetry install --extras "llms-llama-cpp ui"
and continue adding any more it throws at you. for example, the next error i got said ImportError: Local dependencies not found, install with `poetry install --extras embeddings-huggingface`
so my next command was poetry install --extras "llms-llama-cpp ui embeddings-huggingface"
this kept going until the command that got it working was poetry install --extras "llms-llama-cpp ui vector-stores-qdrant embeddings-huggingface"
try to just keep adding the dependency names to the install command until it works
Thanks dude! This helps me a lot
YOOO, IT WORKS: poetry install --extras "llms-llama-cpp ui vector-stores-qdrant embeddings-huggingface"
Thank you, in the cmd it says:
uvicorn.error - Uvicorn running on 0.0.0.0:8001 (Press CTRL+C to quit)
just change it to 127.0.0.1:8001
after adding the extras. is cuda working? do you see BLAS = 1 ?. Everyting runs on cpu on WSL
this line also got me going! thank you:
poetry install --extras "llms-llama-cpp ui vector-stores-qdrant embeddings-huggingface"
This might help as well, this resolved my issue.
set PGPT_PROFILES=ollama
poetry install --extras "ui llms-sagemaker embeddings-sagemaker vector-stores-qdrant"
poetry run python -m uvicorn private_gpt.main:app --reload --port 8001
Impressive. I loaded the instruction manual for my HAAS VF4SS mill (24meg PDF) and asked multiple questions about g-code. It gave correct answers every time.
I input some code and asked for a description of what the machine would do if I ran that code. Very detailed and correct answer. I am impressed.
Thank you.
P.S. I'm running the Mistral LLM and it's only taking up about 7gig of VRAM and is pretty fast with the answers.
in boosting the speed, did you use the Nvidia method or openai key?
Glad to hear, thanks for the feedback. There is a new video that offloads the LLM and GPU's to Ollama. Have a look on the channel if you want to check out the new version. Thanks for reaching out. Updated video PrivateGPT. ruclips.net/video/Ww2oelybDz8/видео.html
Very Strong, Thanks Buddy
Pleasure, glad you up and running. Thanks for reaching out.
I built this up for both the recommended ollama and llama. Ollama didnt seem to utilize much if any of the GPU and documents worked but were too slow to be useful. Llamma works much better. I am sure the ollama issues will be resolved and long term may turn out to be the better method but as of today the llama profile works better
Hi, thanks for the feedback! I dont know if you checked out the new version running on Ollama as backend.
Hi,
I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
Heads up, things may have changed since this video was released.
Would you be able to recreate the process for installing privateGPT again?
Done, Thanks! Link to the new video below....
I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
Video is still valid up until 6:07. A lot of what comes after that has changed in the project. You are going to have to go to the project docs and follow the instructions there for the remainder of the install. You now have more options and it branches after the 6:07 time stamp. If you did everything correctly up until this time stamp and then follow the recommended local,ollama-powered setup you should be fine. When I did this on windows the ollama serve command failed due to an error. You can skip that command and just open the ollama application and proceed with the next step. Note: windows you probably need to break the PGPT_Profiles=ollama make run into two lines. Breakpoint after ollama
Hi, thanks for the feedback. I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
I watched it and it is a good video. I appreciate the work you put into your content. How was your file ingestion with ollama? I couldn't get good performance and had to switch to llama to get it back to reasonable speed.@@stuffaboutstuff4045
Thumbs up man, really great demonstration. Thanks a lot, cuz helped a lot.
Glad to hear it! Thanks for the feedback!
if I have GeForce 1080Ti would it be still worth running CUDA version or rather CPU version on my Ryzen 6 5600X ?
SUCCESS! I managed to get the server running by following the video until 6:07 at which point (as noted in other comments), instead of *poetry install --with ui,local* I used:
*poetry install --extras "llms-llama-cpp ui vector-stores-qdrant embeddings-huggingface"*
After that, carrying on from 9:58 (running locally with an nvidia gpu) I had an error:
"The CUDA Toolkit v12.4 directory '' does not exist. Please verify the CUDA Toolkit is installed properly or define the CudaToolkitDir property to resolve this error."
To fix that I needed to use regedit to add a new key called *InstallDir* under *computer\HKEY_LOCAL_MACHINE\SOFTWARE\NVIDIA Corporation\GPU Computing Toolkit\CUDA\v12.4* (or whichever is your cuda toolkit version) and set the value to *C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4* (this location is your cuda_path which can be found in environment variables within control panel).
Hi,
I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
Thanks for the feedback above!
Awsome...got it working awome video
Thanks for the feedback! If you are interested updated build video released today.
I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
Thank u😮❤
You're welcome 😊 BTW did you check out the latest video on installing Private GPT
ruclips.net/video/Ww2oelybDz8/видео.html
@@stuffaboutstuff4045 I will check it soon, thanks a lot buddy
I I tried it last week and it worked great. Now i got a graphics card and wanted to reinstall it. Unfortunately I keep getting an error that I haven't been able to fix for hours.:
(privategpt) PS C:\pgpt\privateGPT> poetry install --with ui,local
Group(s) not found: local (via --with), ui (via --with)
what am I doing wrong?
I'm trying to get this installed too as I type, but I believe the instuctions on this video are already outdated. It looks like the developer changed the syntax and added more options. I got past this point in the instructions with the following command based on the installation instructions on the privateGPT documentation website:
poetry install --extras “ui llms-ollama embeddings-ollama vector-stores-qdrant”
It then installed numerous libraries similar to the video, but then gave me an error on the next step, but the documentation website says I need to set an environment variable, install Ollama, and download a model, which I haven't done yet, so that is probably the cause of my error.
Hi,
It looks like the install procedure was updated. I will look at making updated video to support the new install options..
@@stuffaboutstuff4045 I've too strucked at the same part can't install 'poetry install --with ui,local'... Help me out !
ok thanks
@@stuffaboutstuff4045
This works. Thanks
Glad the video helped out! Thanks for the feedback..
Thank you. tired of looking for normal instructions
Thanks for reaching out. Glad the video helped!
How solve:
(privategpt) C:\pgpt\privateGPT>poetry install --with ui,local
Group(s) not found: local (via --with), ui (via --with)
(privategpt) C:\pgpt\privateGPT>
I did all the steps to "(privategpt) C:\pgpt\privateGPT>poetry install --with ui,local" with no error
Hi,
I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
is it possible to "train" the private gpt by connection to own datasource/database of existing app, or document in own NAS ?
how?
Hi,
Have you had a look at the updated video I posted.
ruclips.net/video/Ww2oelybDz8/видео.html
You can have a look at the ingestion possibilities here.
docs.privategpt.dev/manual/document-management/ingestion
thank you for the explantation ! can we change the inteface design ?
Pleasure! Updated version should you want to check it out...
I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
please help ! 6:33 , following everything step by step, but after >poetry install --with ui,local it says Group(s) not found: local (via --with), ui (via --with)
I changed "poetry install --with ui,local" to "poetry install --extras "ui vector-stores-qdrant llms-ollama embeddings-huggingface"" (found on page - Private GPT Docs) and it started installing, hope the rest will go smoothly
Damn it ! it doesn't work.... it has bunch of errors after >>>>>>>>>>>>>>> poetry run python scripts/setup.....
Hi,
It looks like the install procedure was updated. I will look at making updated video to support the new install options..
@stuffaboutstuff4045 yes please, that would help a lot 🍻
@stuffaboutstuff4045: if you could type the neccasary new commands in a post
First, I tried to install with Python 3.1.9, but it did not work. With 3.1.7, it went well...
Hi, did you perhaps check out the updated PrivateGPT build on the channel. Let me know if the below video helps get you up and running.
ruclips.net/video/Ww2oelybDz8/видео.htmlsi=6wj-QoaPAfsZjMiM
"Access to model mistralai/Mistral-7B-Instruct-v0.2 is restricted. You must be authenticated to access it."
Hi, thanks for reaching out. Please have a look at the updated build video on the channel. You can have a look at the link below.
ruclips.net/video/Ww2oelybDz8/видео.html
Thank you
You're welcome, glad the video helped. Thanks for the feedback.
I got an error after: poetry run python -m private_gpt
it stopped generating, did not give me the website and I got this: ImportError: Local dependencies not found, install with `poetry install --extras llms-llama-cpp`
is there any chance to get it working?
File "C:\pgpt\privateGPT\private_gpt\components\llm\llm_component.py", line 37, in __init__
raise ImportError(
ImportError: Local dependencies not found, install with `poetry install --extras llms-llama-cpp` >whole error
exactly the same problem arises for me too !!! if you found the solution to fix it Please let me know bruhh
@@sibeejeyachandran8174 @danda3688 you need to continue adding the dependency errors it throws at you into the install command. since you cant install llms-llama-cpp, here is what you would use
poetry install --extras "llms-llama-cpp ui"
and continue adding any more it throws at you. for example, the next error i got said ImportError: Local dependencies not found, install with `poetry install --extras embeddings-huggingface`
so my next command was poetry install --extras "llms-llama-cpp ui embeddings-huggingface"
this kept going until the command that got it working was poetry install --extras "llms-llama-cpp ui vector-stores-qdrant embeddings-huggingface"
try to just keep adding the dependency names to the install command until it works
@@desperbambooThank you so much for your guidance ✌️
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
Thank you so much! It took me a couple of tries, but following your instructions carefully proved successful! I just ran into an issue with CUDA when running the command:
$env:CMAKE_ARGS='-DLLAMA_CUBLAS=on'; poetry run pip install --force-reinstall --no-cache-dir llama-cpp-python
The filename, directory name, or volume label syntax is incorrect.
Hi, thanks for reaching out. Are you running this command in an Admin mode Anaconda PowerShell Terminal.
7:21 I dealt with the previous error reading your comment, but this one... Installing what it says in cmd doesn't do anything.
return callable(*full_args, **dependencies)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\pgpt\privateGPT\private_gpt\components\llm\llm_component.py", line 37, in __init__
raise ImportError(
ImportError: Local dependencies not found, install with `poetry install --extras llms-llama-cpp`
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
(privategpt) C:\pgpt\private-gpt>poetry install --with ui,local
Group(s) not found: local (via --with), ui (via --with)
plz help
use this command instead poetry install --extras "ui"
@MyPc-rw3fb ok, I'll try. Thanks
@@MyPc-rw3fb thank you this command worked for me when I got that not found error
Hi,
I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
@stuffaboutstuff4045 thanks man ill try it out
Hello,
While executing the last code in order to open the app, doesn't showing something.
Hi,
I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
It works thanks for the sharing. Just a question...is it possible to increase/force the use of the GPU... mine only uses aroung/no more than 20% of the GPU capacity?
Hi,
Looks like there is a new build again with a lot more options. I will look at building it the weekend. If you want you can check the new options available.
docs.privategpt.dev/installation/getting-started/main-concepts
@@stuffaboutstuff4045 I've reseted the PC and done a new build, and it's quite fast now, although still not all PC resources are being used (CPU and GPU). The only change made now was:
Before:
poetry install --with ui
Now:
poetry install --extras "ui llms-llama-cpp embeddings-huggingface vector-stores-qdrant"
File "C:\Users\INTEL\anaconda3\envs\privateGPT\Lib\site-packages\llama_index\llms\llama_cpp\base.py", line 104, in __init__
raise ValueError(
ValueError: Provided model path does not exist. Please check the path or provide a model_url to download.
Hi,
I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
when i type this,
poetry install --with ui,local
i will get this,
Group(s) not found: local (via --with), ui (via --with)
Hi,
I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
hi, how to remove or uninstall the mistral 7b later? please advise.
Hi, did you perhaps check out the updated PrivateGPT build on the channel. The newer edition you can completely offload the local processing.
ruclips.net/video/Ww2oelybDz8/видео.htmlsi=6wj-QoaPAfsZjMiM
@@stuffaboutstuff4045 not local processing. how to remove and free up space on my SSD ?
Improving performance using openai API key is not working on my PC. It does not give me answers when I try to ask questions. What do I do? Should I generate another API key?
Hi, It looks like there is a new build again with a lot more options. I will look at building it the weekend. If you want you can check the new options available.
docs.privategpt.dev/installation/getting-started/main-concepts
Pipx not installing on Win10 (64bit). After I enter command " pip install pipx ", nothing happens and it reverts to " C:\WINDOWS\System32> " and awaits new commands.
Hi, difficult one it should not do nothing. I assume you are in a privileged mode command prompt? if you run pipx is it not maybe installed already. What happens when you run pipx ensurepath. Is your pip working? What happens when you try and upgrade pip does it return anything? python -m pip install --upgrade pip. Let me know and I will try and assist..
not sure if theirs more i need to do followed step by step and pipx isn't recognized as a command in cmd.
me too
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
I get an Erro:
poetry install --with ui,local
Group(s) not found: local (via --with), ui (via --with)
instead of --with use --extras
was moved to extras from some weird reason
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
6:39 after typing (privategpt) C: \pgpt \privateGPT>poetry install --with ui, local
I’m getting this error Group(5) not found: local (via --with), ui (via --with)
what should I do
just use "poetry install --extras ui" and follow the advice from the @DesperBamBoo comment
Hi,
I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
Hi.
The part i typed (git clone https) it says git is not recognised. What should i do??
Hi,
I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Build the latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
Following the tutorial to the letter, fails at:
Building wheel for llama-cpp-python (pyproject.toml) ... /
ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects
same error here!!
Hi, these build errors are usually due to missing required SW or privileges. To resolve privilege issues run all commands in Admin mode terminals. Make sure your Python is within 3.11.x This is important. Installed and added to path. Install VS community and make sure you selected Desktop Dev with C++ and Universal Windows platform development. As confirmed make sure all steps are follwed from 3:52 in the video. Let me know if any of the above resolved the issue..
6:33 I've got an error:
(privategpt) C:\pgpt\privateGPT>poetry install --with ui,local
Group(s) not found: local (via --with), ui (via --with)
Use poetry install --extras "ui embeddings-huggingface llms-llama-cpp vector-stores-qdrant"
This is due to a update to the extra yaml files
@@sanstheskeleton-lo4zjthat works
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
Doesn't using openAI's model defeat the purpose of an offline 'Private' gpt session? Otherwise.. why would you need an API key? Maybe the authentication is safe once you verify but it's still a little sketchy to my ignorance.
Hi, I think at this stages it helps out when someone has technical constraints on CPU's and GPU's. Agreed the whole idea is to run offline but I think there are scenarios where using OpenAI comes in handy. Thanks for the feedback!
My system keeps saying im on Python 12.0 and wont complete the install :( . Any work around? I tried removing my version and installing 11.5.
Hi, thanks for making contact. Yes it must be Python 3.11.x. I picked this up as well. Try using the version I have in the video description. Let me know if resolved...
Issue with
poetry install --with ui,local
Error: Group(s) not found: local (via --with), ui (via --with)
Solution: ????????????????
do:
poetry install --extras "ui vector-stores-qdrant llms-ollama embeddings-huggingface"
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
I have installed PrivateGPT on my machine, it worked fine and I am able to ask questions as well.. once I closed cmd. I don't know how to again lunch PrivateGPT. You have any idea how to relaunch the PrivateGPT in cmd or uvicorn? Thanks in advance
Hi, you must just make sure you reactivate the Conda environment again that you created for it (if using conda) and then run the launch command.
OPENAI_API_KEY ENTER-YOUR-KEY sk-z52... how to findout Setx key?
You need to be more emphatic about python's version: IT DOESN'T WORK WITH PYTHON 3.12.
Hi, thanks for reaching out. Please have a look at the updated build video on the channel. You can have a look at the link below. Yes, you have to use 3.11.xx. I think I make it more clear in the new video.
ruclips.net/video/Ww2oelybDz8/видео.html
Hopefully someone can help with this.
So I installed everything no errors and private gpt worked fine until I installed the Cuda stuff to get GPU support. Now I get these errors:
When trying to do the 'conda activate privategpt' I get 'conda is not recognized as an internal or external command, operable program or batch file.
Then when trying to run private gpt with 'poetry run python -m private_gpt' I get this:
C:\JGPT\privateGPT>poetry run python -m private_gpt
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "C:\JGPT\privateGPT\private_gpt\__main__.py", line 3, in
import uvicorn
ModuleNotFoundError: No module named 'uvicorn'
Any ideas? Thanks.
did you clone the folder?
Hi, Make sure you run the command in a conda terminal and not a command prompt window. Otherwise conda is not in your windows path anymore. Let me know if resolved?
@stuffaboutstuff4045 Yes this worked lol. Thank you. Can't believe I was in the wrong command prompt.🙂
I see the error says, 'poetry' is not recognized as an internal or external command, operable program or batch file. - what to do?
Hi, sounds like poetry might not be globally available. Check 3:47 into the video. Did you configure it globally by following the steps from there exactly? From the bit of upgrading pipx in a admin command prompt (Not Conda) up to running pipx ensurepath. Let me know if this shines light on your issue. Thanks for reaching out!
@@stuffaboutstuff4045 I re-try and multiple times and stil give me the same error. How do I get it to recognized the command? Please help🙏
Hi, did you see this reply? Hi, sounds like poetry might not be globally available. Check 3:47 into the video. Did you configure it globally by following the steps from there exactly? From the bit of upgrading pipx in a admin command prompt (Not Conda) up to running pipx ensurepath. Let me know if this shines light on your issue. Thanks for reaching out! Please confirm if those were followed...? Let me know if resolved or if you still stuck with the error.
you can install pipx and poetry again after you activate the local environment
also close your conda prompt and open a new conda prompt
Get the llama.cpp wheel error on windows?
Hi, did you run in admin mode prompts? Correct version of Python 3.11.x. That is what usually would cause a build fail.
I am stuck on the GPU install step:
$env:CMAKE_ARGS='-DLLAMA_CUBLAS=on'; poetry run pip install --force-reinstall --no-cache-dir llama-cpp-python
The filename, directory name, or volume label syntax is incorrect.
Any help is much appreciated
removed the CMAKE the rest ran but still very slow so I am unsure if it is running on my GPU. Any way to check. Thanks again for the amazing walkthrough!
Hi, glad you got the environment variable loaded!
Hi, You can open your task manager or resource manager on windows. Execute a search in PrivateGPT and see what takes the hit the CPU or GPU. One of them should spike and show you where the processing is being handled.
@@Bbentley81 I re installed CUDA, And Cmake ...that worked
@@oniseikeji6023 that worked! thank you! cheers!
Everything was done, but page 8001 did not open
Hi, can you verify the webserver starting in your terminal when you launch Private GPT. Have a look from 7:25 in the video. Let me know..
@@stuffaboutstuff4045i have tha same problem, i even changed it to localhost:8001 still dnt open
20:24:18.638 [INFO ] private_gpt.ui.ui - Mounting the gradio UI, at path=/
20:24:18.689 [INFO ] uvicorn.error - Started server process [6736]
20:24:18.689 [INFO ] uvicorn.error - Waiting for application startup.
20:24:18.690 [INFO ] uvicorn.error - Application startup complete.
20:24:18.692 [INFO ] uvicorn.error - Uvicorn running on 0.0.0.0:8001 (Press CTRL+C to quit)
found it : //127.0.0.1:8001/
I was unable to access the host ":8001". Then I realized that it is necessary for Anaconda PowerShell to remain open. Correct?
o comando: poetry install --extras "ui"
Hi, can I suggest you check out the new video on the below link. Let me know if you are up and running.
ruclips.net/video/Ww2oelybDz8/видео.html
My computer
i3
16ram
1g viga
Any privategpt not work
Hi, is it starting? Did you configure it for GPU, CPU or OpenAI?
How do I turn it on? Cpu
If you follow all the steps in the video up to 7:00 mark it should be configure for local LLM and CPU. Did you make sure all steps were followed up to that point? Incl setting it up as local and executing the scripts to download the LLM.
do not follow this tutorial - it is sadly outdated and does not work. check rest of comments to verify
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
This doesn't work now. According to your own comment below the install process has changed and you'll need to make a new video. Waste of time!
Hi, I created an updated PrivateGPT video, Welcome to check it out. PrivateGPT with Ollama support. Latest version 0.4.0.
ruclips.net/video/Ww2oelybDz8/видео.html
Thank you so much I appreciate.
Hi this project is working on my pc but 1 thing is not missing which is "LLM chat" why it is not responding" the answer I have download the model:
TheBloke/Llama-2-7B-Chat-GGUF
llama-2-7b-chat.Q4_K_M.gguf
but not working i have changed the name in setting.yaml and setting-local.yaml in it.
llamacpp:
llm_hf_repo_id: TheBloke/Llama-2-7B-Chat-GGUF
llm_hf_model_file: llama-2-7b-chat.Q4_K_M.gguf?
please tell me the solution or any video reference tutorial
Thank you so much in advance.
Hi, just checking in to see if you were able to resolve this issue. I have released an updated video to build the latest version correctly. You can check out PrivateGPT v0.4.x using this link. Let me know if you are sorted and got it up and running correctly. Thanks for reaching out. ruclips.net/video/Ww2oelybDz8/видео.html
Hi, Thanks v4 was very very helpful for me. Now
Please make the video on that we could find the specific text to draw the live rectangular bounding boxes on doc with accuracy where the text locate in page or line... Regex may also be use in that but please make video on it.
Thank you.
Hi, Can you help me with this error? I have downloaded the tool kit and it is able to find it, But still it gives this error.
Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.3/include (found version "12.3.107")
-- cuBLAS found
CMake Error at C:/Windows/System32/privateGPT/myenv/Lib/site-packages/cmake/data/share/cmake-3.28/Modules/CMakeDetermineCompilerId.cmake:529 (message):
No CUDA toolset found.
It might be trying to use the VS Build tools. Can you check if this article is applicable to your problem.
stackoverflow.com/questions/56636714/cuda-compile-problems-on-windows-cmake-error-no-cuda-toolset-found