Sir, can u explain clearly about after creating model file in which terminal we need give command ollama.create -fmodelfile qwen2.5-large:7b .And after installing docker where to give command docker-compose --profile development up.clarify my doubt sir....
many thanks for the great video. one question: where is the LLM downloaded to? i want to make space again to try another LLM. how can i delete the 4GB again?
I have this problem. Linux and rtx4090 where I can run qwen32b model at full speed (GPU) via Ollama in terminal but as soon as I run it via Bolt, it run on CPU instead at snail speed. VRAM goes to 22gb usage but no GPU utilization .
To run 32B models effectively, you need at least 64 GB of RAM, though 128 GB is recommended for smooth performance. In your case, the issue happens because the model exceeds your RAM capacity, forcing the system to use virtual memory (swap) on your SSD, which is significantly slower than RAM. This delay occurs before the data reaches the GPU. While your GPU (RTX 4090) is powerful and its VRAM is utilized, the bottleneck lies in RAM and swap processing. So try upgrading your ram 😀
for some reason my instance defaults to anthropic no matter if i select ollama, the reason i discoverd that is that the anthropic key was not set and it complained about it in the error output even when ollama models was selected.
@@MervinPraisonthanks for reply, do i need to change modelfile or any of the commands if i change model? also shouold i increase more than you have set?
I think this is a good start. But, it is still not that powerful and for someone who already codes fairly quick. I feel like this is much slower at the moment, still. Give it a couple years and I reckon this might be worth.
Hey, I saw your videos. They're great and informative but your thumbnails are not appealing enough. I think you should hire a Professional Thumbnail Artist for your videos to increase your view count cause every impression matters. I can improve your ctr from 2-3% to 15%. Please acknowledge and share your contact details to get your thumbnail.
este es el tipo de video y contenido el cual ahi que dale no me gusta por el simple echo que no fue claro con la explicacion ahi cosas que no se logran hacer como la parte del modelfile la tercera linea como la puso entre otros detalles este es un video que esta echo solo para monetizar y ganar dinero no para enseñar realmente
I don't understand anything since the start.... You say "In your terminal" but what terminal? Dude you can't start a video tutorial by assuming certain things
![Twenty One Pilots - “The Line” (from Arcane Season 2) [Official Music Video]](http://i.ytimg.com/vi/E2Rj2gQAyPA/mqdefault.jpg)
Yo. You just make the tedious process of the trifles into a juicy episode which is amazing for long term style for RUclips algorithm
Thank you for your contributions Mervin! Including a repo link in description would be helpful, just saying
Love your videos - nice work on showing warts and all
I like this approach. Thank you Mervin
Great job, well done!
This is awesome!!!
Hi dear, What is your computer specification?
Thanks man i managed to set mine up. I am struggling to add addiotional api Keys se help
good video as always.. but i think you missed out the creation of the .env.local file
He assumed that you already know to use ollama and env tweaks.
se salto muchas cosas
Hey Mervin you forgot to tell your viewers to change the .env file to use Ollama local API
Yes I want to know how to set .env for ollama
exacto ahi que desle no me gusta al video para que haga las cosas bien como debe ser no solo hacer el video para el monetizar
@@Luis-Traders 😃
@3:16 when i select Ollama nothing there.. what would be the issue? i believe it is related to bolt-ai-dev warning pull access denied for bolt...
Sir, can u explain clearly about after creating model file in which terminal we need give command ollama.create -fmodelfile qwen2.5-large:7b .And after installing docker where to give command docker-compose --profile development up.clarify my doubt sir....
docker compose command is run in bolt.new-any-llm folder that you cloned from github
Nice :D
many thanks for the great video.
one question: where is the LLM downloaded to? i want to make space again to try another LLM. how can i delete the 4GB again?
ollama rm
In Mac the files are hidden have to command shift . I think. Or command shift R
it works perfect but GPU is a must to have the speed. for me when i asked , it corrected some files
you should have given him credit!
well why not use open router api key to test the local bolt. there you can find the big models for free to use and the bigger context length as well
Can you show, how Can we use other LLM not only qwen2.5-large:7b?
muchas gracias
Thanks
Hello do you perhaps know how I can import an existing project into Bolt.new locally so that I can continue working on it?
Ollama API Key:
Not set (will still work if set in .env file)
I have this problem. Linux and rtx4090 where I can run qwen32b model at full speed (GPU) via Ollama in terminal but as soon as I run it via Bolt, it run on CPU instead at snail speed. VRAM goes to 22gb usage but no GPU utilization .
To run 32B models effectively, you need at least 64 GB of RAM, though 128 GB is recommended for smooth performance. In your case, the issue happens because the model exceeds your RAM capacity, forcing the system to use virtual memory (swap) on your SSD, which is significantly slower than RAM. This delay occurs before the data reaches the GPU. While your GPU (RTX 4090) is powerful and its VRAM is utilized, the bottleneck lies in RAM and swap processing. So try upgrading your ram 😀
why the hell when you install not asking fro API key but when i don it is?
same prob
@MervinPraison how did you get cursor to suggest you lines of code in the terminal?
How to know the context max length for each parameters?
I have used gemini-1.5-pro but now also not working Gemini has context length of 2M there is some problem with the software only
What is the best llm for coding apps?
Just can't get ollama models to appear under ollama. 2 hours of diagnosing with claude and still nothing. everything appears to be running.
same issue
Why don't any tutorials explain anything about the part of calling the APIs to make the model work, always skipping that important part?
for some reason my instance defaults to anthropic no matter if i select ollama, the reason i discoverd that is that the anthropic key was not set and it complained about it in the error output even when ollama models was selected.
You need to configure the .env.local ollama local endpoint
Can this generates also backend or only frontend? If only frontend it's a waste of time.
Can I also use openai api for this application?
how can I reach in this "terminal"?
do we need graphics card for this?
Is bolt.new using claude by default?
Where can I search modelfile
ive done everything several times but im not getting the actual code and files
Did you try increasing context length ?
Also try various models
@@MervinPraisonthanks for reply, do i need to change modelfile or any of the commands if i change model? also shouold i increase more than you have set?
I think this is a good start. But, it is still not that powerful and for someone who already codes fairly quick. I feel like this is much slower at the moment, still. Give it a couple years and I reckon this might be worth.
how do you make the modelfile?? what file does it has to be? I have no clue how to make this in my CMD prompt. ( windows computer )
Right click and create a new file. Name it as modelfile
@MervinPraison Thanks. But what kind of format file do I need to make. Just a folder, txt, ,?
@@mikevanaerle9779
Create modelfile.txt and run below command
ollama create -f modelfile.txt qwen2.5-large:7b
Doc: mer.vin/2024/11/bolt-new-ollama/
@@MervinPraison Thank you
@@MervinPraison for me it does not see the preview, or code in the right
To run more than 7B , do will we need more RAM, right? 64gb or more?
Yep
25 GB ram
I'm running @16gb ddr4's 2x, i5 11thG 114k, 2.70Ghz/4-6gbGpu, 6C-12Thrd, my laptop handles 7b, 8b, 11b & 16b. Any higher starts to slooow down lol.
Can I install this from within VS terminal?
Yes, You can use any terminal.
@ oh great this really looks interesting will try to install it. Thanks.
DOES THIS WORK ON WINDOWS?
Yes
what all are we talking here? API?
This fork of bolt.new enables the use of any provider including local (on machine) inference provided by Ollama as in this example.
Hey, I saw your videos. They're great and informative but your thumbnails are not appealing enough. I think you should hire a Professional Thumbnail Artist for your videos to increase your view count cause every impression matters. I can improve your ctr from 2-3% to 15%. Please acknowledge and share your contact details to get your thumbnail.
este es el tipo de video y contenido el cual ahi que dale no me gusta por el simple echo que no fue claro con la explicacion ahi cosas que no se logran hacer como la parte del modelfile la tercera linea como la puso entre otros detalles este es un video que esta echo solo para monetizar y ganar dinero no para enseñar realmente
Could please again you can do proper video explanation , this is not not good explanation sir
Literally nobody is explaining it proper for people like us to understand. No one on youtube. Smh
I don't understand anything since the start.... You say "In your terminal" but what terminal? Dude you can't start a video tutorial by assuming certain things
Visual studio code terminal or your preferred ide. You are cloning the GitHub repository
You can't expect the tutorial starts with an explanation on how to turn your computer on.
@@zipaJopa you must be the funniest person at home.. this youtuber didn't even share the git clone command as he claimed in his video..
@@ShadowDoggie but it's bolt.new-any-llm?
Terminal / Shell / Console / Command Line all mean the same.