It's like 0.0014 per prompt just get it lol things unbelievable I was stuck for 3 days with clide and chatgpt and cascade. All of them. Could t handle the size of my algo any more. Thjs thing did what I was trying to figure out in three days. Did it in 3 prompts
Deepseek r1, definitely the full model you can access via web/api, is really difficult to host on your own hardware. You'll need to start with a smaller model like Phi-3.5/4 or a smaller Llama model.
The important distinction here would be that this guide shows you how to use LocalAI with Docker. Similarly, you can use Ollama with Docker. Docker is just meant to give you an easy one-command way to start an environment consistently. So I'm going to take the liberty to answer the difference between Ollama and LocalAI, because those are actually the systems you are interested in comparing. For large language models, Ollama is actually just as good if not even better supported by the community. You can use Ollama just fine in Docker as well: hub.docker.com/r/ollama/ollama However, LocalAI supports more types of AI models than just language models in one package. I aim to show more of those possibilities later on hence I focus on LocalAI. If you just want to use language models, definitely try out Ollama!
I always start my prompting with the question “Was 2024 a leap year”. On simple local machines it’s telling you fairy tales. That’s disappointing. But thanks for your good explanation 👏
phi...ok but not that powerful, i thought there might be something i don't know in this video...i would rather use openwebui or anythingllm with ollama if you just need a chat interface
That's right, I will show in later content that LocalAI allows you to use other models not just LLMs. If you just want a language model, you can use ollama
Will try it surely!
Hey, thanks for the video.
Kindly make a video how to setup and make own deepseek r1 api
It's like 0.0014 per prompt just get it lol things unbelievable I was stuck for 3 days with clide and chatgpt and cascade. All of them. Could t handle the size of my algo any more. Thjs thing did what I was trying to figure out in three days. Did it in 3 prompts
Deepseek r1, definitely the full model you can access via web/api, is really difficult to host on your own hardware. You'll need to start with a smaller model like Phi-3.5/4 or a smaller Llama model.
What's the difference between using docker and ollama? If you could give a detailed explanation, I'd like that
The important distinction here would be that this guide shows you how to use LocalAI with Docker. Similarly, you can use Ollama with Docker. Docker is just meant to give you an easy one-command way to start an environment consistently.
So I'm going to take the liberty to answer the difference between Ollama and LocalAI, because those are actually the systems you are interested in comparing. For large language models, Ollama is actually just as good if not even better supported by the community. You can use Ollama just fine in Docker as well: hub.docker.com/r/ollama/ollama
However, LocalAI supports more types of AI models than just language models in one package. I aim to show more of those possibilities later on hence I focus on LocalAI. If you just want to use language models, definitely try out Ollama!
@zenvanriel oh, LocalAI. I understand now. Thanks
Awesome content, could you make a video where you customize a reasoning model It further like connecting It to a folder of PDF files as database
This is indeed all in the backlog of the next videos! Stay tuned.
How does this model perform on your device? Let me know!
nice name zen
We share the same first name based on your handle? If so, love it!
@ yes we do 🤝🏼
The voice is not synchronised with your video
Oops, looks like it is slightly off indeed. Will make it better next time.
I always start my prompting with the question “Was 2024 a leap year”. On simple local machines it’s telling you fairy tales. That’s disappointing. But thanks for your good explanation 👏
Yeah, for local devices a RAG implementation makes more sense. I'll explain that later on.
How can i contact u zen
In a few weeks I'll make sure we can all get in touch. Stay tuned.
phi...ok but not that powerful, i thought there might be something i don't know in this video...i would rather use openwebui or anythingllm with ollama if you just need a chat interface
That's right, I will show in later content that LocalAI allows you to use other models not just LLMs. If you just want a language model, you can use ollama
How can you make an interface or make it receive input files?
There is a opensource webuis
This is upcoming in a next video!