Private AI Chatbot on Your Computer - Step by Step Tutorial
HTML-код
- Опубликовано: 30 сен 2024
- Here is a simple 5 step process for installing and running a local AI chatbot on your computer, completely for free. I made this for non technical people, and you should be able to follow along step by step.
Step 1: Install Ollama ollama.com/
Step 2: Copy and Paste Llama 3 install command using Terminal
Step 3: Add other LLM models (optional) ollama.com/lib...
Step 4: Install Docker www.docker.com/
Step 5: Install OpenWebUI docs.openwebui...
Login to OpenWebUI and start using your local AI chatbot.
We just released a brand new course on our E-Learning platform that dives much deeper into running a private ChatGPT alternative on your computer.
You can learn more here: bit.ly/skillleap
Get access to our new Private AI chatbot course as well as twenty other AI courses here: bit.ly/skillleap
I am on windows 11. I was having issues with it not loading anything when I typed a question. It was thinking and that was it. I found that after I went into the Admin Panel > Settings > Connections and turned off (OpenAI API) it started working. Just in case anyone else has this same issue.
may I know what do you mean by admin panel??
I have this setup behinds a VPN on my local machine at home and it works like a charm on my phone remotely!
Any tut on how to do that?
Can the ip for the machine this is running on be shared to others on the same internal network? I assume so... but just making sure before I give this a try.. Also, is there a way to train these models from here or no? Thanks for this BTW.. subbed.
Since this doesn't like running on potatoes, how do i uninstall this thing, do i just uninstall docker and ollama?
Inside of the settings of OpenWebUI, you can uninstall each LLM you installed first. Then you just uninstall docker and ollama.
Loved this video! I will be leveraging your instructions with my students as I introduce AI into their course content!
Thank you! really like the simple step-by-step approach in an easy to follow ways. Managed to successfully set this up! Subscribed to your channel to support you!
Thank you
turn off the wifi to make sure this data isnt going anywhere...until you reconnect to the internet.
Hello, does this Private GPT have something similar to ChatGPT’s custom instructions feature, or can I create separate specialty setups for statistics, Authoring, and/or Coding (COBOL, C#, Fortran, etc.)?
Can writing (authoring ) samples of mine be uploaded for specific chat sessions?
Thank you. At last a tutorial about setting this up, that worked for me. I have zero experience with docker, and now I have it running on my computer! I'd like to learn even more about the usses of docker (just for hobby purposes at home).
Thank you so much for this incredibly helpful tutorial! I really appreciate you taking the time to walk through all the steps in detail. Running a private AI chatbot on my own computer is going to be a game-changer. Can't wait to dive in and try it out!
I just installed the 7B 4 GB model, very lightweight and useful in grid down scenarios.
Very well made video for anyone to get started quickly 😊👍🏻
This is great! Thanks for sharing!!!
Can it connect to sharepoint sites to read from documents on the sites?
Downloaded and working perfectly. Thanks.
Would I be able to use this functionality (or is there a better approach) to review/summarize/respond to specific, esoteric websites that are not typically included in the LLM databases? Could I provide specific URLs and ask for detailed feedback on what's found at the given URL?
Thank you for this easy-to-follow tutorial. How would you deinstall a model that is to big to work with your hardware equipment?
Very thorough breakdown, thank you!
I just bought an HP laptop the day I saw this video. I'm stuck after downloading llama... it won't open after download for me to continue...
Thank you very much for this comprehensive and handy tutorial.
My pleasure
It was really easy to install, can you show how you use this with gcolab, using ngrok as a tunnel, and dockdesktop running locally and ngrok endpoint being the openWEBUI? This would allow everyone to use it on mobile as well, and also include setup to talk directly with the different models that support it, and explain how uncensored modell can effectively help you in ways that chatGPT cant..
Super helpful tutorial! Worked like a charm! Question - I'm on a windows machine. What are the BASE specs and BEST specs for a windows machine to make this process as fast/efficient as possible? Thank you. Thank you. Thank you.
Completely private chat client inside spying Chrome inside spying macOS. Nice.
Would this functionality allow me to upload screenshots of content and have the LLM be able to read through and respond to what it sees in those images?
what kind of laptop hardware do we need? will it work with normal office thinkpad without a high end gaming gpu for the laptop?
It works with a lot of laptops. But the speed will vary. I think if you have a gaming gpu, it should be decent speed. The big speed issue I noticed is if I upload a lot of files to my knowledge base
yes any laptop with gaming gpu on it , ai loves nvidia gaming gpu even if you have 4gb vram like me , im able to run ai models but make sure if you have less then 8gb to download 1.5b and 3b and 7b other then that you 'll see alot of problems .
It took a few tries but I finally got the Linux items and Vm to run.. Could not find the VM setting in the bios without a few videos and one finally showed on a mother board close enough to mine and poof, one more restart and docker is running Llama3!!! Thank You!
I enjoyed your video. thanks and kudos. What about updates? Should we uninstall and reinstall for an update?
Can you talk with Ai ,your plans day by day in this programme
If i host this in a private server for my team .. will my private files and data i used for this llm gets stored or trained by the models of the other person using the llm other than me and my team..?
error 500????
How can I share this on a home server for family members to access?
I have terminal on my phone, so actually it would work then to
Great video,never even knew about this stuff! Only problem, my computer memory isn't large enough. Where do i even start?
If your computer doesn’t have the specs to handle it, you’ll be limited to using this on meta.ai. Running it private requires a beefy computer right now
So interesting thing just happened ...Chatbot got math wrong when I was trying to ask it an easy conversion of how much of a 3qt bag of activated charcoal to add to a 2.6 pound bag of coco coir/perlite 79/30 mix to get a 70/30/10 mix ..the answer is the entire bag ...first it kept thinking it was 2.6lbs of coir and perlite each ...but the big thing was it tried to tell me 3 quarts is equal to 2.086 pounds ...the. it admitted it made a mistake...I didn't think thay was possible?
I’ve heard language models aren’t good at math. it’s a language model not a math model
how beefy should my computer be?
Thank you very much for such a clear and concise guide. I wish to use this to analyse confidential interview transcripts as a way to check my own manual analysis and interpretation of the data and this needs to be done offline so this is perfect. Please can you suggest if there is a model that would be best for this kind of analysis? BTW watching you use terminal took me back to my youth using MS DOS!
Llama 3 or Mixtral are the two models I recommend. You can use both at the same time and compare results
Im so frustrated... when i run the app after intalling it, nothing happens! nothing opens! what is going on?
I believe installing these apps turns your computer into a remote zombie that receives instructions to run a web based network. So in short your power is contributing to the network with out your knowledge. This is what bitcoin is doing… people think their computer was solving math problems but in reality they were part of a huge network that’s creating value with bitcoin…. I tried telling people but nobody understands how information works.. if you ever get bored look at what info is being shared from your computer without your knowledge…. Now imagine that times 1k! I have a computer that will turn on I the middle of the night if I sneeze… thats how sensitive it is.
Great video! Can I make this work using Perplexity?
No it won’t work with that one.
Meta and Privacy don't go hand in hand.
before I take your course: can I use this in a server environment, so all of my employees can work with the chatbotˋ?
I don’t believe you can with this set up. The only LLM options I know for server based environments are the name brands like Gemini, and ChatGPT on azure OpenAI service. I’m sure a developer can set this type of thing up on a private server but I don’t get that far in my course
thank you so much sir🌸
is it possible to use this uncensored?
Docker doesnt work on older macs.
What’s the limit to the number of documents you can upload? Is there a limit on size or count?
I don’t see any limit but it hasn’t worked for me with more than 10. It lets me upload them but doesn’t give me useful responses. I’ll see if they have documentation on this.
When I open the web UI along with my local LLM Lllama 2, I can see GPT-4 and GPT-4o in the choose LLM dropdown. Do you know what is causing this?
If you add OpenAI api under settings, those appear in the menu. That’s the only way I know of
the API key added itself automatically. It somehow picked the api key from somewhere.I didn't do anything. 😂 I was searching for this so that I could host an LLM locally and do organization-related tasks. Now I'm having doubts.
Would you make a video about interact with documents, Pls
You got it
Very very good. Thank you.
Windows Powershell update requires installs alternate powershell.
Does Window Terminal work the same? I'm going to make a new video tomorrow with my Windows. My main computer is a Mac and the documentation said it was the same process.
Ok, so once everything is install and you play with it and it all works, what do you do the next time you want to run it?
Let's say, my computer has been turned off and then on again. What do I need to run to start the ai again? Just docker? I would like instructions in how to restart everything. What are the step?
Launch docker and click on the port to launch it. You can also press stop on docker to stop it
@@SkillLeapAI Thank you! Just got Docker up and running.
Alright, everything is up and running! Unfortunately on my 2019 ryzen apu even phi3 is slow as molasses. Oh well, I thought that might be the case.
I have to give you great props on this tutorial. I've seen a number of these olama-webui install tutorials and your tutorial was the most clear. The best!
Thanks for your help!
Thanks for this great video!
can u upload pics?
is the MacBook Air m3 10 core gpu good enough for the last thing?
Yea I have M3 and have had no issues yet
@@SkillLeapAI thanks a lot
How do you get the files for knowledge base . If yiu want your bot to focus on certain things . Or overall knowlwdge
the knowledge base if your own documentation. Like blog posts your wrote, company policy, etc. It already has knowledge without your own documentations.
@@SkillLeapAI thank you. Ok so I need to research my own knowledge base. I'm guessing perplexity is good for that or any other
Windows only ?
Why wouldnt you use something like LM Studio instead of doing this via localhost and a docker instance?
That's a very good option too. I was more familiar with this option and it's the version I've set up on my computer.
can you connect stable diffusion to lm studio
Install a ton of apps to get the functionality. Why would I not go directly to chatgpt?
Privacy. Lot of people can’t just upload documents to ChatGPT. It’s either for privacy, work rules, sensitive info, etc. this is local and private. You’re not sharing anything with any company
I made this video for those people that need a private solution due to work rules or privacy concerns
Of course it’s easy to follow your good step by step instructions, but the real magic is as usual happening behind the curtains… 🪄 So how’s ones privacy guaranteed and kept private in the long run…?
Not even close to the same install/setup process on Win 11 PC.
I’ll run it on my windows machine tomorrow
I have it running in windows 10.. It's so easy even I did it in minutes and wrote half a book of 20 chapters. Life is simple when you think and believe it is simple.
Not for M3…
I have M3 mac
@@SkillLeapAI interesting it errored out and said something about the intel chipset not compatible. I’ll have to retry again
When installing it from the Docker website, choose Apple silicon instead of intel.
You are amazing!
Thank you!
@@SkillLeapAI Yw, keep making these!
A "Windows Preview" version that only installs Ollama over and over again. Thanks for catering to Mackies only. UNsubscribed.
Are you a troll. I installed it on windows 10 easily. Running perfect and fast.
Using the same steps, I successfully installed and run it on Windows too. Hope this helps.
@@ProfessorCrumbs65 Yes, Win10, but not Win11. Doesn't work for me either
“you just lost an important customer” ahh
Oh, man , final solution!