Amazing video, Llama 3 seems way better than expected, I hope Meta would release 8 B for code, and 8 B for Python, I don't think Meta would do something like 400 B for code but that would crazy if they did, even as is Llama 3 would help the community a lot.
Thank you. Yeah I find it hard to imagine them making the 400B open, it would change the open source community. Also an 8B for code will be a beast for local use.
Great set of benchmark questions! I had some surprising results when I tried out the question about Sally's sisters. Oddly enough, Llama 3-70B missed it, while Llama 3-8B nailed it every time. Both GPT-4 Turbo and GPT-3.5 Turbo got it right as well. Interestingly, Clade 3 Opus got it mixed up and said she has two sisters. Really intriguing to see the differences!
It's very surprising for Llama3-8B to answer correctly when 70B can't. I just tested it again in another platform (Meta.ai) where we have the 70B model and it's worked just fine. Where did you test the models? was the 8B llama hosted locally?
Yeah I used llama3-70B in the video and I tried the 8B one in my machine as well, sadly the smaller models are no match to the bigger ones no matter how much fine tuning we make. this is why it's important to have specialized models (Llama3-8B coding for example) and that model will be good at a specific thing
Amazing video, Llama 3 seems way better than expected, I hope Meta would release 8 B for code, and 8 B for Python, I don't think Meta would do something like 400 B for code but that would crazy if they did, even as is Llama 3 would help the community a lot.
Thank you.
Yeah I find it hard to imagine them making the 400B open, it would change the open source community.
Also an 8B for code will be a beast for local use.
Great set of benchmark questions! I had some surprising results when I tried out the question about Sally's sisters. Oddly enough, Llama 3-70B missed it, while Llama 3-8B nailed it every time. Both GPT-4 Turbo and GPT-3.5 Turbo got it right as well. Interestingly, Clade 3 Opus got it mixed up and said she has two sisters. Really intriguing to see the differences!
It's very surprising for Llama3-8B to answer correctly when 70B can't.
I just tested it again in another platform (Meta.ai) where we have the 70B model and it's worked just fine.
Where did you test the models? was the 8B llama hosted locally?
The 7B models are pretty shitty tbh :(
They are fast but wrong most of the times.
Yeah I used llama3-70B in the video and I tried the 8B one in my machine as well, sadly the smaller models are no match to the bigger ones no matter how much fine tuning we make.
this is why it's important to have specialized models (Llama3-8B coding for example) and that model will be good at a specific thing
nop it s not better that GPT
you're right now it's not, these models are beating each other like there is no tomorrow, to this date GPT-4o is the one at the top.
@@redamarzouk before gpt 4 omni, gpt 4 turbo still better, the only best point with llama is free model :)