Видео

Hi, I'm sorry but no, I did not encounter a similar issue. Maybe reach the Google support to be helped 🤷 Thanks a lot for this kind comment ^^

Great advice!

De rieng ^^ Profite bien de ta Machine Virtuelle

Its the smaller model for fair comparison. You wouldn't compare it with o1

Yes, it is true. But the released was for the Gemini 2.0 flash. I needed to compare it against equal size/price models as @nashh600 correctly pointed out.

We need to select a persona for Gemini ?

@ I don’t know Gemini, but the others, I would strongly recommend it to get a better result.

Oh okay, I didn't knew that. I thought it cannot do multi turn because it's single turn only in the QwQ Space ^^ Thanks a lot for the precision !

Yes, tetris is quite difficult for LLMs. Only Claude 3.5 Sonnet and Qwen2.5 Coder 32B got it right on my tests. Even gpt4o didn't got it in my test (but i think it has more related to luck)

I mean, you can't fit the required 24 gb of VRAM on your graphics card, but hey, only one way to find out if it works right.

@@hatnis well, it was free to ask 😅

Yes, but you will have to offload a lot in your CPU/RAM. It will run pretty slow but it will work 👍

In the video, I ran it in my 24Go of VRAM. I think it is q4_k_m

I was able to get it working on my new Mac mini base m4 pro chip model. QwQ-32B-Preview-GGUF bartowski repo. IQ3_XS quantization. the only one I could download as this one is 13.71 gb of ram. Note because I am using a Mac mini apples ram is unified so my 24gb of ram is shared between the gpu and cpu. if I spent spent a extra 300$ from the 1.4k I spent for the m4 pro model I could of loaded the max quantization model but I don't really do AI locally as I use online Ai services more. I hope this helps!

Yeah, I wasn't expecting 3.1 to be THAT bad neither 😅

Ohhh that's awesome man !!! Thanks a lot ^^

I liked and subscribed you vidéos too 😉

🤣🤣🤣🤣

Thanks a lot man 🙏👍👍

I'm using Groq, they have specialized LPUs (specialized hardware for LLM inference) that allows insane speeds and it is free to use (and you even have an api). There is also sambanova that offers the same thing

France and you ?

24go at the very least (with insane quantization like q2-q3). You absolutly need at least 1 3090/4090. I would recommand running it in your RAM with CPU and have at least 64Go of RAM and run it as q4-q5

unquantized it is about 130go.

But you can use it for free on Groq and SambaNova

@@volkovolko I'm going to run it with 8GB vram and 32GB ddr4 ram as usual. Q2 is around 26 Gigs so it shouldn't go to swap

I'd highly doubt you'd be able to run it with that. I have a RTX 3070 and 48GB of DDR4 RAM, and I tried running Llama 3.2 11b vision and it doesn't even load.

I would be happy to test with any prompt you give to me ^^

Hummm, I don't really understand 😅😅

Oui en soit c'est possible, c'est simplement que la majorité de mon audience est anglaise

@@volkovolko mouais.., non mais c'est super intéressant, ce qui serait cool aussi ça serait de comparer les different types: GGUF, AWQ, GPTQ, bon après on comprend que c'est toujours une manière de rationaliser les virgules flottantes (et donc la quantité de bits et donc la taille du modèle), mais je pense que les gros progrès de l'IA maintenant ça va être sur la quantization, réussir à condenser au maximum les modèles pour les rendre, certes un peu moins précis mais beaucoup plus pertinent (utilisables sans avoir des racks de GPU). Enfin, en tout cas merci.

Je partage tout à fait cet avis

Yes, I had some issues with Clipchamp, which I use to edit the video. But I'm glad it helped you !!

Ici : www.microsoft.com/fr-fr/software-download/windows11 ^^

Quite verbose, explain exactly what features you want, well ordered and formated, use prompting techniques like Chain of Thought etc. In practice, I first ask chatgpt or any other LLM to create a good prompt for me then, I regive it to him. I gives way better results

I don't even know what it is 😅😂😂

Yes it is !!

They the issue is that we need to balance the complexity of the tasks. If it's too easy all models get it right so we cannot compare them If it's too difficult all models fails so we cannot compare them. Tetris and Pac man games seems currently a good fit for SOTA and aren't that tested so that's why I use them

Oui, cependant il y a souvent des options "par défaut". Si je lui demande de me créer un programme en python, il va systématiquement me faire un hello world. Donc certains détails "par défaut" on pourrait penser que tu n'as pas besoin de les donner (comme ce serait le cas pour un humain) mais en réalité ca améliore la qualité de la réponse. Cependant, je pense qu'à l'avenir on aura pas besoin de préciser ces chses "Par défauts"

Yeah, before this test I thought that nowadays we don't need good prompting anymore. Especially due to the system message prompting of ChatGPT, the fact that it ask the model to reformulated the question before anwsering However it turns out that I was completely wrong

Yes, you're right. Providing a better, longer and more precise prompt would have surely improve the quality of the LLM answer. However, the video would have been longer and I think in real life, most people are lazy and don't provide to the LLM a well done prompt. Maybe I will make a video comparing how big is the performance hit between a good and a bad prompt

Yes, they are 4bits as recommanded by Ollama. However, I can make a new video trying with 8bits as you asked. But I think the results will be pretty much the same. Maybe I should rather make a video comparing each quant of the 7b

@volkovolko yes in my experiments with llms so far it looks like for small models (<8b params) quantization does have an impact on model output quality while for larger models the impact is less noticeable.

I'm trying to do my best. When I made this video. I didn't had any speakers so I couldn't test the audio nor make great cuts

😂😂😂

I do real software projects as I'm a developer. While Claude and GPT4o are still better for big projects, qwen is a good alternative for just little prompting to avoid going to stack overflow for quick and simple questions

Glad you liked it!

For you a 3B Qwen can be better for discord bot coding than GPT4o ? (not a judgment, I just want to know)

@volkovolko Yeah, it does a lot better at importing packages, using Python properly, and even does well at managing errors gracefully.

Yes, the results of the tests I made in this video seems to show that : GPT4o < Qwen2.5coder32b < Claude 3.5 Sonnet (new)

The point is to compare quality... simple as that. Once you know quality, you can consider other factors like speed, price, availability, and of course confidentiality. The fact that Qwen2.5-Coder-32B is even close to Claude while being a _small_ open-weight model is amazing. Of course other factors can matter more than just quality. Speed and price are just as important. But limiting it to "Only compare quality when technical specs are comparable" makes no sense.

@@sthobvious actually makes sense because if you think to compare gpt-3.5 and gpt-o1 or gpt-4o, do you really think this is fair? Gpt-3.5: 😭 Gpt-4o & gpt-o1: 🗿🗿

Yes, it's so awesome they this technology is going toward open sourcing 👍

Okay, I will try in the next video

As explained in the video, I'm looking for other original tests. If you have one that you want me to try in a following video feel free to leave it in a comment so that I can try it in the following video

@@volkovolko If you are testing how to write a snake game, then you are basically testing knowledge retrieval, because that code exists in 1000 variants on the Internet. It gets interesting if you demand variations, like 'but the snake grows in both directions' or 'random obstacles appear and disappear after some time in not too close proximity of the snakes'. Think of whatever you want, but if you can do Tetris or snake is hardly a test for llms these days.

@5m5tj5wg The 'better' model is not one that can retrieve known solutions better, but the one that can piece the solution to a unheard but related problem better. If you can find the question and the answer on the net then comparing a model with 32B params to a Multi-hundred-billion parameter model like GPT4o or sonnet makes even less sense, because of cause they can store more knowledge. You need to ask for solutions to problems where you cannot find the answer on the Internet to evaluate how good a model will be in practical use.

Yes, there is a part of true. However, I think you can all agree that you don't want a 50+ min video. Also most of the code you will ask it to make in the real world will also be knowledge retrieval. As developper we very often have to remake what as already been made. And the Snake game isn't that easy for LLMs. The Tetris game is very difficult and I didn't ever see a first try fully working

And it is interresting to see that the Qwen model did better on these "retrieval" questions than GPT and Anthropic despite being way smaller in terms of parameters. It indicates that knowledge can still be compress a lot more than what we thought

Okay, I will try in the next video

Yeah, it might be because of the luck factor. Or maybe the architecture of qwen is optimised for high quantizations levels 🤷‍♂️ Or maybe your q8 version wasn't properly quantized, I think they updated their weight at one moment

luck it's called temperature nowadays :D

Yeah, I now. Top_k also right ? ​@@66_meme_99

Yeah, for me the win was for Qwen. But okay, for the following videos, I will always let one second chance for all models. I will soon make a video comparing each size of qwen2.5 coder (so 0.5B vs 1.5B vs 3B vs 7B vs 14B vs 32b) So subscribe if you want be notified ^^ I also started to quantize each model in GGUF and EXL2 on HuggingFace for those who are interested : huggingface.co/Volko76

Seems very interesting I will try it tomorrow, for me nemotron 70b was the best but even on my 4090 I can't run it locally.

I made the video comparing sizes : ruclips.net/video/WPziCratbpc/видео.htmlsi=o3eKo-3pGY78wmMr

Yes, 70B is still a bit too much for consumer grade GPUs

Ben non pourquoi ?

Asus ROG STRIX 3090 32Go ddr4 3200MHz i9 11900kf

Yeah, I totaly agree. The Qwen series (especially the coding one for me) are just so amazing. I don't know why they aren't as known as the llama ones.

Do you want me to make a video comparing the 3B to the 32B ?

​@@volkovolko Yeah, that would be really cool to see! I'd love to see how the models perform.

Okay, I will try to do it tomorrow

Ah mdrrr ben ecoute tant mieux, j'aime bien sa voix xD

?

@ Germany countryhuman: me singing

Wtf ?

@ Germany countryhuman: do support me

Il suffit d'installer la version 4 (qui est en alpha) de ollama : github.com/ollama/ollama/releases/download/v0.4.0-rc5/ollama-windows-amd64.zip Ensuite tu peux installer llama3.2 et utiliser des images. Si tu le souhaites sinon tu peux créer un rag (mais c'est plus complexe)

Yes, it is. ChatGPT is also an LLM

Ahh ça fait toujours vachement plaisir d'aider les gens 🙏🙏