How does this compare to GPT 4o and Sonnet 3.5 (New)? I have an M1 Max 10cpu/24gpu 64 RAM…do you think those specs could run this? Any tutorials for how to set this up?
I only use Cursor and Claude for coding and Claude is definitely better than ChatGPT and Qwen in my opinion. I don't know your computer may run it or not but you can basically install the LM Studio and try.
@@MrFluke039 If you have a powerful machine and if you don’t want to pay for a paid service then it’s why you want to run a local LLM. Reason why you use Ngrok is because Cursor cannot access to your local LLMs. It does everything on their server so the LLM model should be on a server too.
@@johnbean3429 Sure it does, but if you noticed the model he selected was GGUF in a purple badge next to model’s title not MLX with the grey badge At 0:02
I only tried for JS and I can’t say it’s better but it’s close to Claude and ChatGPT quality. Also keep in mind the model may respond better on different programming languages.
@@daburritoda2255 I only tried the GGUF version and the Mac was on fire after a couple of minutes chat with Qwen. I think you even need a more powerful machine.
How does this compare to GPT 4o and Sonnet 3.5 (New)? I have an M1 Max 10cpu/24gpu 64 RAM…do you think those specs could run this? Any tutorials for how to set this up?
I only use Cursor and Claude for coding and Claude is definitely better than ChatGPT and Qwen in my opinion. I don't know your computer may run it or not but you can basically install the LM Studio and try.
@ Why do you prefer Claude over GPT for programming? Are there certain use cases/quality that you’ve seen you like better? Have you tried o1 mini?
Why i can not load model this qwen 2.5 32b when i downloaded
What tool do you use for these kind of demos ? Thank you for sharing by the way. Can't wait to get my mac too 👍
@@molmock Thank you. It’s Screen Studio.
@ozgrozer thank you !
What are your Screenstudio export settings? Looks great!
@@Businessology.I don’t do any custom settings. Just choosing this background image.
Why do you need ngrok? Doesn't localhost work directly?
Cursor does everything on their server so it can’t connect to localhost.
i have question why running llm locally why u still need to host it on ngrok to put it on plugin ?
i am not ai dev so i am not sure how it work sorry for dumb question
@@MrFluke039 If you have a powerful machine and if you don’t want to pay for a paid service then it’s why you want to run a local LLM. Reason why you use Ngrok is because Cursor cannot access to your local LLMs. It does everything on their server so the LLM model should be on a server too.
If you try the MLX it would be much faster
@@raadkasem I’d like to try it if I find an easy way to install it
LM Studio recently added support for MLX, so I believe the demo in the video is already using MLX.
@@johnbean3429
Sure it does, but if you noticed the model he selected was GGUF in a purple badge next to model’s title not MLX with the grey badge
At 0:02
@@johnbean3429 where is there a freaking video of it then on youtube can't find like pls.
qwen qodar
성능은 어떠한가요? 클로드나 GPT4o 보다 낫다고 보시나요?
I only tried for JS and I can’t say it’s better but it’s close to Claude and ChatGPT quality. Also keep in mind the model may respond better on different programming languages.
@@ozgrozer Thanks for your reply :)
Is your ram 64gb?
No it's 36GB. I sped up the video on model response because it was too boring to watch. But the speed is mostly 12 tok/sec.
@@ozgrozer I have 24gb m4 pro, I don't think I have enough memory to run it as LM studio was giving me errors
@@daburritoda2255 You could try Ollama too. But even if you could run, it would be very slow to respond I guess.
@@ozgrozer yeah i did, no luck 🙁, have you tried running the MLX version? if so how does it differ from the GGUF quantisation
@@daburritoda2255 I only tried the GGUF version and the Mac was on fire after a couple of minutes chat with Qwen. I think you even need a more powerful machine.
Hmm too bad ngrok is not open source
There are some open source alternatives but I think Ngrok is the most popular one