Using Ollama To Build a FULLY LOCAL "ChatGPT Clone"
HTML-код
- Опубликовано: 14 май 2024
- In this video, I show you how to use Ollama to build an entirely local, open-source version of ChatGPT from scratch. Plus, you can run many models simultaneously using Ollama, which opens up a world of possibilities.
Enjoy :)
Join My Newsletter for Regular AI Updates 👇🏼
www.matthewberman.com
Need AI Consulting? ✅
forwardfuture.ai/
Rent a GPU (MassedCompute) 🚀
bit.ly/matthew-berman-youtube
USE CODE "MatthewBerman" for 50% discount
My Links 🔗
👉🏻 Subscribe: / @matthew_berman
👉🏻 Twitter: / matthewberman
👉🏻 Discord: / discord
👉🏻 Patreon: / matthewberman
Media/Sponsorship Inquiries 📈
bit.ly/44TC45V
Links:
Code From Video - gist.github.com/mberman84/a12...
Ollama - ollama.ai/ 
Наука
Ollama series! This was a great starting video❤ thank you for all your hard work
This is awesome! I'd love to see more. I feel like this can become something pretty robust with enough time.
Every time I need something, you present a tool doing exactly that. Thanks!
Glad to hear it!
Thank you for this video; after trying many models, and failing, I finally succeeded at running a local GPT! 🤗
How do you like the responses of your local GPT compared to official one?
This was done so! Perfectly. Every part swollen with meaning
Ollama is incredible! Runs fast LLMs. And i see in your channel about autogen and so... agents building and find that i was looking for. I love your channel and your teaching manner. Thanks Mattew!
You created an incredible video, precise, concise, and I couldn't have asked for more!
This is the type of straightforward high quality content ❤
This is so cool! And the fact that they give you an rest endpoint running on a port on the machine is great.
Thanks! This concise video is exactly what I was looking for to help me take next steps with Ollama!
Thank you so much Mathew, this is so incredible!
You're so welcome!
Really great video! The easiest way to get history is to take the `context` which was given in the response and just pass it back as the 'context' field in the request.
Wow, this makes it so extremely accessible. Your video also shows how accessible interacting with these ai's is in general as well. I haven't programmed much since I was younger, but have been wanting to, and this seems like a great jumping off point! Now I just need to wait until the Windows version comes out.
MADA SAKA
I'm building my own personal AI assistant but every time I start something a week later something better drops. My god, this is impossible. I've got to think better about my abstractions to make some of this stuff more drop-in ready. That might be an interesting video (or series of videos) for you Matthew, if not likely a bit advanced for your audience.
I'm in the same boat. The tech changes so quickly, my ideas become antiquated as soon as I get something working 😆
The nice thing is if you stick with using OpenAI API, that seems to be the standard
@@matthew_berman I have an Azure account and I'm trying to use it to act as a server for the different models rather than hosting them locally. I'm having so much trouble doing that because the models that are included with Azure aren't the ones I want to try out. Do you have any advice?
You're lucky. I still have to learn Python. But since ChatGPT is developing too fast, when I learn, my knowledge would be obsolete, because just now we can create a personal assistant using GPTs very easily, do you agree ? 🙏👍
@@DihelsonMendoncame too, once I focus on something then later I find something else exists and with high quality than the previous one hhhh
This is so cool! I'd love to see a deeper dive.
You are a god send. Thank you
Ive been using it through WSL for windows
Wow. That's the best 10 minutes I have spent today. Great learning.
Great video! Simple. clear and concise. Thanks for that. An idea for a continuation (as a complete novice on AI) could be how to start a simple training on the model to keep improving on some topic we would like?
You are so passionate. And you are right to do so. Thanks !
Another great video, I was able to achieve the same in LM Studio running multiple models, on Mac, by spawning instances from the CLI and incrementing the port. Then in my autogen app passing different llm_config objects to the specific assistant agent.
Nice video! You definitely picked up a new subscriber here. I’d be interested in seeing how to build out a RAG solution with Ollama, and also how to make it run in parallel for multiple concurrent requests.
Great Video.. Thank you!. I would love to see a deep dive into using Olama with Autogen, Having each agent use its own model.
Great content bro, you're my new favorite youtuber!
Love your videos, much respect and appreciation for all the work you do. I do have one humble suggestion, if you could hide your image just enough to see what you have typed, for instance at 8:49, it would have been great. I know that most RUclips instructors do this, not sure why but please take that into consideration. Either way, thank you for all you bring.
Thanks for this nice video. I would like to see a video about MemGPT implementing the history function instead of just pasting everything in front of a new prompt.
A good idea could be: PrivateGPT with Huggingfaces model cards in it is passed the prompt with the task to tell the best model for that prompt. Then the prompt is passed via ollama to that model with MemGPT on top of each model. That actually might be the most powerful local solution right now.
Integrating Ollama and Canopy would be a great video. Having that local retrieval would have many use cases.
the video on Ollama is really beautiful. Among other things, I would also start doing benchmarks on the various text generation user interfaces. Ollama allows me, for example, to use my laptop with a small GTX 1060 and Dolphin at incredible speed. the same laptop struggles with Oobabooga. However, after some interactions, the model goes into "overload", as if the RAM is no longer enough. In short, this comment is a too long thank you for your excellent work. And a hope for more videos about Ollama and local models.
@Matthew_berman: You are very brilliant! I have been watching ollama videos but none of them taughthow to use it with API or structured it the way you did. Keep it coming bro. Thank you so much. God bless!
About the privateGPT, I found the accuracy can be improved if the database change from duckDB to elasticsearch.
Absolutely the best video yet. ollama looks amazing.
Now show me what options there are for doing similar such things in android apps :)
this model will allow us to make open source models fast, I love the simultaneous part, please make more tutorials on this once it hits windows without wsl
agree... This OLlama really looks promising Matthew!! Light weight and simple. More plz!!
I am at WSL now, join me. WSL - Windows Subsystem for Linux. It is at Microsoft Ignite. The title is How to install Linux on Windows with WSL. So, you are on your own now. I have several computer requiring updating. good luck.
@@chrismachabee3128 you are an AI generated comment. Please follow terms of service on RUclips for automated accounts, creator of this bot.
This was a fantastic video ! Cheers for that !
I did comment about it few weeks ago on one of your videos ! Indeed, very useful for autogen (but also for Langchain).
Very interesting, would love to see how well it works with autogen or any of the other multi-agent libraries. Looks like you can import any gguf as well.
yes! it would be really interesting how autogen + Ollama goes !😍
I'm interested, too! I wonder if Autogen is obsolete now that OpenAI unleashed the kraken with the GPTs! What do you think?
@@BibopGresta1i think you have yourself a popular follow-up video, given the comments asking about autogen 😉
@@BibopGresta1 Unlikely, GPT's are more of a single custom Agent instead of a set of agents working together.
Awesome video, they have a WIndow version now (3-30-24), and it installed an ran perfectly.
DUDE! This was an awsome tutorial!
Amazing job!!! Everyone wants more!!
😮 Please create a video integrating Ollama with autogen!
and autogen's function calling with local models too!!
This was my first thought. Please do this
Yes!!! Do this with AutoGen!
Great idea dudes 🔥🔥🔥🔥🔥
Easy enough! I’ll make a video for it.
Many thanks it's really useful and really easy because you explain extremely good
This is just great and easy as well! Could you show us how to train these models with PDFs and Websites?
This was so Dope! - I have been using Ollama for a while, testing multiple models, and because of my lack of coding expertise, I had no understanding that it could be coded this way. I would like to see if you can use Ollama, memGPT, and Autogen, all working together 100% locally to choose the best model for a problem or question, call the model and get the result, and then permanently remember what is important about the conversation... I Double Dare You. ;)
+1
So mind blowing~! Thanks Dude~!
Another cool video! I hope that they come up with a windows version soon :) Definitely want the deeper dive. ty
Awesome work!!!
Great video, just a quick note, you actually do not need to all the previous messages and responses as the prompt, the API response contains an array of numbers called the context, just send that in the data of the next request
Thank you very much! 🤝
loved this, Matthew! Right to the point, super hands on. This looks like an awesome project!
Great vid!
super video! if you can make something more deeply about memory management, it would be lovely.
Please make more videos on these topics in detail.
00:01 Building Open-Source ChatGPT using Olama
01:27 Ollama and Mistol enable running multiple models simultaneously with blazing fast speed.
02:50 Running multiple models simultaneously with Open-Source ChatGPT is mind-blowing.
04:14 Building Open-Source ChatGPT From Scratch
05:40 Creating a new python file called main.py to generate a completion.
07:00 Adjusting the code to get the desired response and adding a Gradio front end.
08:35 Built an open-source ChatGPT from scratch using Mistol
09:56 The conversation history is appended to the prompt in order to generate a response.
Thanks for talking about fully local engines. Do you have a video with hardware recommendations for this?
This is so over my head! But I'm following! Very cool!
Wow! Jaw dropping video!
I hope this works as well when they release it for Windows! Switching between models so fast like that is crazy!
Hi matthew,
You are doing amazing work to teach everyone about real power of AI with support of LLM
I have a question , what to do if we to build something which works with any kind of documents as like this video model are working does it possible to do such things as well and what if we able to build them is there any way that we can deploy them in production as website or applications
is there any way please make a video on it
i'm looking forward to it
thank you!!!!!
And boom goes the dynamite.
I'll bet integrating this with autogen isn't hard. Heck, you coukd just ask autogen to re-write its own interaction settings to use the various models.
The interesting bit here would be asking autogen or the main dispatch model to find the best answerable model based on the context of the prompt.
As always, great vid!
Please make a guide on setting it up on the virtual machine, and creating API so we can use it in our apps (even with Make for example)
I am not a coder, but somehow I achieved this building. Million Thanks!!
Its a bit slow, but good enough to showcase to friends.
By anychange we can host this live ? If yes, then How, kindly make a video on that !!!
Million Thanks again😀🙏
This video is awesome❤
Thanks I find it. Great help
Oh my god, what amazing video.
Hey Matthew!
Great video. Please help me with this, would hosting fine-tuned open source models on Sagemaker cost lesser as compared to GPT-4 API? Is there a comparison anywhere on any forum, reddit, etc? I want to fine-tune a model on my data, and I am thinking of going with GPT-3.5-turbo fine-tuning, but it's really expensive at scale. I want to know how do fine-tuned open source models compare to these prices (assuming we get a good efficiency at our desired task after fine-tuning)?
Would really appreciate any thoughts on this. Thanks a lot!
Building an AutoGen application using Ollama would be wonderful ! Example: one of the agents is a coder, implemented by a LLM specialized in coding etc.
I was about to write this myself.
haha me too!
I started writing same comment, and then saw yours :D
I'm an autogen application using ollama, I was going to write this comment too.
@@MungeParty O nice to meet you! Why autogen ollama app is interested in this? :D
Boy, Matthew is so inspiring. Thank you for ruining my weekend plan. I'd interested in the same matter as @padonker: how can we train with own data?
This is amazing. I live in terminal and I do python. perfect!
Thank you!!
Congratulations, great video, I wonder if I could install a model similar as Claude 2 ( obviouslly if there's a similar that I could install on Ollama) and train it with documents (doc or pdf in Spanish) to create a webchat for questions and answers.
Thanks!
It will be awesem to have tutrial about how to create fine tunend model from i.e. mistral to gguf running with ollama :)
I consider like so other subscribers you could create a video integrating ollama and autogen and the conversation can be stored on database and another video creating a AI personal assistant
Really awesome Matthew !!
I have a request: Can you make a video for a free LLM that can interact with Big Data like AWS Redshift please?
i think the only thing it needs now is to be able to monitor a project folder so you can reference a set of documents. then I could ask it to help me with my specific project and not waste time and tokens feeding it code.
Great Thank you very much.
Nice. Now I understand why chatbots only allow a few prompts before they start over. They fill up their context window. BTW, it would be great to ad RAG with document and Google search. There's also a way to access Ollama from Siri. That would be ideal.
cant wait for the autogen expert video!!!
Hi! Thanks for the content, I always follow your videos. Can you show how to deploy ollama on runpod to have this multi-model setup running on cloud?
Thanks for making it approachable. How would this work with Docker? And a portable nvme drive?
Can you take a deep dive into using the Modelfiles to make your own model for specialty takes? Where can we find out things like token limits?
The Orca2 language model got the killers question right. When you first ask the question, you may disagree with it's answer, but it justifies itself and does correctly answer the question as asked.
So excited last night forgot my manners, if its possible Mr Berman, I would really like to see models talking to each other via there dialogue windows. say by adding a conversation starter window to set the topic and seeing there path of there conversational logic. Please. (Teams of separate modals processing a given task)
Regarding the memory issue, can you integrate this with Memgpt? Could you please make a video for that?
Thank you for the info. Kindly, let us know what are the specs on your pc? I have a very slow response on my macbook air from 8GB Memory and CPU of M1
Finally, a good video🥳
Great show "Yes dive deeper" Link them working together bi-directional communication. How far can it go.
wow, fantastic. OpenSource models and ecosystem is everyday more powerful
Ollama FTW! ✌
Hi, what are the minimal specs for some of the most popular models ? Is there any model which can ran on 4GB RAM and a slower 2-cores CU, like an i3 ?
Are the models that you can pull quantized or should we still get our models from TheBloke?
awesome, going to check text to sql for my project, hope it gives proper sql query for give schema. what do you think?
is there an api end point that i can use, as openai's api replacement?
Hi Matt, as someone who watches every video, I'm feeling overwhelmed and am wondering if you could do a "take a step back" episode every once in a while -- where you go over previous content from a broader perspective. For example, what is the difference between LM Studio, Ollama, Jan, AnythingLLM etc and where should someone start? Or go over the "gotchas" and frustrations in the comment sections to highlight those little errors and solutions commentators found but may have been missed by the casual viewer. It would be a review of old content, but with updated fixes, comparisons, and general perspective/advice. Thanks!
Interesting! Will consider
Hi Matthew. Can you make a video about the various LLM models and the specialization of each?
Ollama is cool if you are looking to build a personal assistant on your own PC. If you try to hit a model with multiple requests, be prepared to wait in line.
Please make a video on the new vision support of open interpreter!
Hi! Excellent work!I tried to replicate but when visiting the URL the input prompt tab to the right top is missing...
Great video! I'm trying to figure out where the model is downloaded. What do I search for? I'm on a mac and since my terminal is at the root (~) level I assume it should be there. But if I do ls -a in that folder, I don't see it. I'm using Mistral, and a search in the finder for Mistral doesn't show it either. Anyone know how to find it?
For the Linux user, I had and issue running the script directly from vs code, so I ran it on a terminal and it's working now, the script it's "python main.py"
how do I modify this code, especially the data that I'm training the model with, for building my own custom faq chatbot?
Please let me know how.
Please, continue development.. Maybe inclusion of local Redis cache on docker and using it for conversion memory?