This is crazy. I've just tested the latest version of TheBloke/Mistral-7B-Instruct-v0.2-GPTQ in localGPT and it gives very meaningful and coherence sources when enabled the --show_sources flag of course. And all the previous models cannot do this and simply giving random junks but not this new model! Not to mention it is blazing fast for such quality response and taking only 6 to 12 seconds using my RTX 3060 12 GB with ~3 GB of vector db size. And the system prompt really affecting the result and the default localGPT system prompt is not good for my use case and I just tweak it a little bit and ask it to double check for its mistake and bam! It seems that it can correct its own mistake! Can't imagine the power of MoE Mixtral-8x7B-Instruct-v0.1 if that small model already really good.
I came up with the killers problem as part of my own benchmark, and I proposed this problem to Matthew Berman to part of his testing benchmark and he began to use it. Now I see it here! Pretty cool to see it out in the wild. I said that when an open source model finally answered this question correctly we will have arrived at the point where open source models are just as good as OpenAI's. This was before GPT 4, even chat gpt could not answer it correctly. GPT4 does now answer it correctly but there was even a period where even it gave the wrong answer.
@@Someone-tc4wu It's a fine-tuning of a base model to operate in instruction/response manner. While the base model operates in auto-complete mode basically.
I like Mistral as a challenger, and a lightweight-ness ambition. Perhaps more of open source could be a path, offering beneficial cloud computing integrations.
Is it correct that Zephyr (in the past) significantly boosted Mistral with a careful training system? Could Zephyr do this again and make a super-super-moe?
Yes sure when gpt-4 is 220m x8 moe and this one just 7x8. By far this is the best open source that can beat gpt-3.5 and not even instructions model yet
@@tatsamui Anything that compares to GPT-3.5 is pretty good. At this point do you think people are figuring out how to game the tests during training? For example, you would not want the prisoner's dilemma on an LLM test anymore because people would train for that specifically.
It's actually open. Free on a service. And now costs nothing. Uncensored version. The largest is going to be an open source version of gpt4. If not tomorrow. Within months. Open source is going to blow past closed by end of 2024.
Need help with Building your LLM powered Applications? Reach out if you need help: calendly.com/engineerprompt/consulting-call
I just entered local LLM operation field. What a time to be alive!
We already have working GGUF files for the MoE. 2024 is, indeed, going to be extremely interesting.
I agree!
hi, which models are currently best for web AI chat bots that are open source, can be fine tuned locally, and run on 8GB vram vps comfortably
I will look at Mistral-7B, Llama-8B and the Qwen models.
This is crazy. I've just tested the latest version of TheBloke/Mistral-7B-Instruct-v0.2-GPTQ in localGPT and it gives very meaningful and coherence sources when enabled the --show_sources flag of course. And all the previous models cannot do this and simply giving random junks but not this new model! Not to mention it is blazing fast for such quality response and taking only 6 to 12 seconds using my RTX 3060 12 GB with ~3 GB of vector db size. And the system prompt really affecting the result and the default localGPT system prompt is not good for my use case and I just tweak it a little bit and ask it to double check for its mistake and bam! It seems that it can correct its own mistake! Can't imagine the power of MoE Mixtral-8x7B-Instruct-v0.1 if that small model already really good.
I came up with the killers problem as part of my own benchmark, and I proposed this problem to Matthew Berman to part of his testing benchmark and he began to use it. Now I see it here! Pretty cool to see it out in the wild. I said that when an open source model finally answered this question correctly we will have arrived at the point where open source models are just as good as OpenAI's. This was before GPT 4, even chat gpt could not answer it correctly. GPT4 does now answer it correctly but there was even a period where even it gave the wrong answer.
It has very strong reasoning capabilities from my tests. I am running the instruct version locally using LLM Studio.
@@TheReferrer72what does instruct model mean ?
@@Someone-tc4wu It's a fine-tuning of a base model to operate in instruction/response manner. While the base model operates in auto-complete mode basically.
I like Mistral as a challenger, and a lightweight-ness ambition. Perhaps more of open source could be a path, offering beneficial cloud computing integrations.
Is it correct that Zephyr (in the past) significantly boosted Mistral with a careful training system? Could Zephyr do this again and make a super-super-moe?
Why not compare to GPT-4? With GPT-3.5 costing just 10% per token it's really a much smaller model and not the leading advancement in LLMs.
Because it wouldn't make headlines then
Yes sure when gpt-4 is 220m x8 moe and this one just 7x8. By far this is the best open source that can beat gpt-3.5 and not even instructions model yet
@@tatsamui Anything that compares to GPT-3.5 is pretty good. At this point do you think people are figuring out how to game the tests during training? For example, you would not want the prisoner's dilemma on an LLM test anymore because people would train for that specifically.
Why compare free AI model to paid service? Are you stupid?
It's actually open. Free on a service. And now costs nothing. Uncensored version. The largest is going to be an open source version of gpt4. If not tomorrow. Within months. Open source is going to blow past closed by end of 2024.
How big a difference on the benchmarks is actually statistically significant?
Love the Mistral team - great job guys!
great job training the Woke lobby
I agree, they are really able to ship. Their speed is amazing.
This restriction was removed
What restriction, please ?
@@DihelsonMendonca the restrictions on using the output to train another model. You're just not allowed to reverse engineering the model now.
Noticed Phi-2 (2.7B) today, anyone tested it or an opinion in comparison?
I hope they make an AI assistant similar to OpenAI's soon with the same tools
excellent report! Many thanks. More api options are great. (more fully open-er source would have been great-er-er)
It's over for OpenAI.
Where does this model stand against the MPT models from MosaicML
This is far better than MPT models
This is awesome! Could you make a tutorial on getting to run on Apple Silicon?
I've try this morning, but it's complicate to run on windows.
I tested it and indeed it's pretty good compared to GPT 3.5 ! unfortunately is has the same stupid censorship which is against human rights
Please recommend me a open source model to ADD EYE BLINK TO THE AVATARS ( REALLY NEED IT :( )
Every new Ai chatbot is labelled as chatgpt killer 😂 … but no one managed to kill it …
It did, beats 3.5 in almost all benchmarks.
Sir, any tips if I using japanese language?
Why do people keep comparing stuff to Llama? It was never a top performer, and it's not lightweight either.
Test against GPT-4. 3.5 is good, but it isn't GPT-4
Maybe 9x7 will be better
Baffling that people think it's better than 3.5...haha. If you just test it with reasoning it soon becomes quite clear it isn't.
It is better than gpt-3.5
Not uncensored sucks