Awesome! I've been using Backyard AI for local LLMs. Obsidian is new to me (switched from Notion) and Ollama looks PERFECT for integrating notes and AI. Thanks for the great video!
Great video. Thank you, brother! I, too, installed Llama 3 on my machine, and the program/machine was so slow, it just seemed to freeze. It would EVENTUALLY eke out a response, but...no. So thanks for the intel about Phi3, especially!
Thank you very much for this video! I just downloaded Obsidian a couple days ago, and was looking for free Copilot alternatives. Do you have any recommendations for the Whisper plugin? (Alternatives, or ways to us a local LLM like in this tutorial)? Thanks again!
Thank you for the video, this is long time on my todo list, but rather have it on homelab server instead of locally, (not sure if possible). Please, what are optimal hw requirements, more cpu, or more memory? What was the bottleneck, It was a bit slow with response locally? Any reason?
Homelab server sounds really cool! Like I said, I'm using 2020 Mac Air m1 so I experienced slow performance when I used a bigger model like llama3. Phi was working great though
What would be awesome is for this to be able to use an Ollama instance running in a container on another machine - that way I can use my container host for Ollama with all it's grunt, and keep the load off my smaller laptop.
@@the-writer-dev i have been experimenting with running Ollama on a Docker container using Proxmox LXC. After configuring the Ollama base URL field with my server's URL on Obsidian copilot, everything works like a charm
@@the-writer-devI will cover the costs, allowing us to remove WebsUI and solely utilize Ollama or LMstudio for the backend. With LMstudio now featuring CLI command capabilities, it's even more beneficial as it reduces the layers above Copilot. I conducted a test with LMstudio's new feature today, and the Copilot responses were noticeably faster on my low-end laptop. Additionally, we can incorporate groq's fast responses and edge neural voices, which are complimentary.
Good video. I get this message in the terminal while setting the server step 4. >>> OLLAMA_ORIGINS=app://obsidian.md* ollama serve The "OLLAMA_ORIGINS" variable in the context provided seems to be a custom configuration, and serving files with `ollama` would again follow standard Node.js practices: 1. To set an environment variable similar to "OLLAMA_ORIGINS", you could do so within your project's JavaScript file or use shell commands (again this is for conceptual purposes): I can connect with you through other channels to work on this step.
It can run CPU-only, it can even run on a Raspberry Pi, it's just going to be slow if you don't have a beefy GPU. Pick a smaller model and it should be alright. But unless you care about being able to customize the model in a few ways or having extra privacy with your chats, it'd probably just be easier to use an external LLM provider
I'm running Llama3 8B model with Ollama on a basic M1 Mac with 16gb RAM - it's snappy. There is no strict cutoff for hardware requirements - if you want to run larger models with less RAM, Ollama can download quantized models which enable this (for a performance tradeoff). If you're on PC with GPU, you need 16GB of VRAM to run Llama3 8B natively. Otherwise you'll need to use a quantized model.
Awesome! I've been using Backyard AI for local LLMs. Obsidian is new to me (switched from Notion) and Ollama looks PERFECT for integrating notes and AI. Thanks for the great video!
Thanks for the support! As a developer and solopreneur, AI and Obsidian are my essential tools so I will keep uploading about them!
Great video. Thank you, brother!
I, too, installed Llama 3 on my machine, and the program/machine was so slow, it just seemed to freeze. It would EVENTUALLY eke out a response, but...no. So thanks for the intel about Phi3, especially!
Excellent video!
Going to try this out.
Thanks and let me know your experience!
how do you get it to read pdf's?
Thank you very much for this video! I just downloaded Obsidian a couple days ago, and was looking for free Copilot alternatives. Do you have any recommendations for the Whisper plugin? (Alternatives, or ways to us a local LLM like in this tutorial)? Thanks again!
Thank you for the video, this is long time on my todo list, but rather have it on homelab server instead of locally, (not sure if possible).
Please, what are optimal hw requirements, more cpu, or more memory? What was the bottleneck, It was a bit slow with response locally? Any reason?
Homelab server sounds really cool! Like I said, I'm using 2020 Mac Air m1 so I experienced slow performance when I used a bigger model like llama3. Phi was working great though
What would be awesome is for this to be able to use an Ollama instance running in a container on another machine - that way I can use my container host for Ollama with all it's grunt, and keep the load off my smaller laptop.
That is an interesting idea..! Thanks for the feedback I will look into this to see it’s possible
@@the-writer-dev i have been experimenting with running Ollama on a Docker container using Proxmox LXC. After configuring the Ollama base URL field with my server's URL on Obsidian copilot, everything works like a charm
Copilot needs integration with Groq AI, and Text to speech integration inside chat room.
That sounds interesting idea!
@@the-writer-devI will cover the costs, allowing us to remove WebsUI and solely utilize Ollama or LMstudio for the backend. With LMstudio now featuring CLI command capabilities, it's even more beneficial as it reduces the layers above Copilot. I conducted a test with LMstudio's new feature today, and the Copilot responses were noticeably faster on my low-end laptop. Additionally, we can incorporate groq's fast responses and edge neural voices, which are complimentary.
Awesome video man! Just sent you connect on LinkedIn
Thanks for the support and I will check it out!
Thanks for the video. If it says the `address is already in use`, run this exact command: osascript -e 'tell app "Ollama" to quit'
says "osascript: command not found"
If you get an error when you try to run the serve command about port already being use then run 'osascript -e 'tell app "Ollama" to quit''
says "osascript: command not found"
Good video. I get this message in the terminal while setting the server step 4. >>> OLLAMA_ORIGINS=app://obsidian.md* ollama serve
The "OLLAMA_ORIGINS" variable in the context provided seems to be a custom
configuration, and serving files with `ollama` would again follow standard
Node.js practices:
1. To set an environment variable similar to "OLLAMA_ORIGINS", you could
do so within your project's JavaScript file or use shell commands (again
this is for conceptual purposes):
I can connect with you through other channels to work on this step.
can you please explain more i cant get mine working
what is the hardware requirements to run models locally.
It can run CPU-only, it can even run on a Raspberry Pi, it's just going to be slow if you don't have a beefy GPU. Pick a smaller model and it should be alright. But unless you care about being able to customize the model in a few ways or having extra privacy with your chats, it'd probably just be easier to use an external LLM provider
I'm running Llama3 8B model with Ollama on a basic M1 Mac with 16gb RAM - it's snappy. There is no strict cutoff for hardware requirements - if you want to run larger models with less RAM, Ollama can download quantized models which enable this (for a performance tradeoff). If you're on PC with GPU, you need 16GB of VRAM to run Llama3 8B natively. Otherwise you'll need to use a quantized model.
Can this work with LM Studio?
Good question I haven’t played with LM studio. I will and let you know!