this is the fastest AI chip in the world: Groq explained
HTML-код
- Опубликовано: 21 фев 2024
- Try Groq: groq.com/
SimTheory with Groq: simtheory.ai
In this video I explain what Groq is, demonstrate low latency with Groq and show how incredibly fast Groq is. Groq will open up whole new possibilities for AI with 25x faster speeds and 20x less cost. I'm excited!
plz sub, like, comment etc. xox
1:37 That "Silicon Wafer" looks suspiciously like a circle of corrugated cardboard covered in tinfoil...
Groq was before Grok
Quick is an understatement. I wonder if we get to a point where the systems actually record our inputs before we press enter. E.g. they are working in the background as we type in our input and when we press enter it may be instantaneous ?
Thanks for the video! So is Groq going to be sold as a chip sometime? Like will it be installed on a motherboard like a GPU or RAM or NVME?
I was thinking about this also - maybe as an add-on PCIe card. For the crowd that may believe that like "not your keys, not your coins" - "not your cloud, not your data". I could see Apple as a candidate for buying them to make their already-fast Neural Engine on the motherboard into something that puts OpenAI out of business.
Current Groq configuration comes with limited stroage space of SRAM and it would not be sufficient for Training but Inference purpose. It 's comparable to Tesla's Dojo but with lesser computation power of Tesla Dojo. However when the potential buyers would have to ponder the limitation before making the purchase decision, Layers of KV Cache and Batch Size would make it only suitable for medium size computation. Not to mention that Complexity of Software should be taken into the consideration.
i feeel like this first chip is more a proof of concept or a marketing effort from Groq more than anything, this video is a perfect exapmle of that.
gotta make stuff for us hype boiz
Hopefully their next chip fixes these shortcomings.
Really good video. Looks very professional and the information was very easily understood. Great work!
It’s crazy though that it’s will cost about $2 million worth of cards to run a 70b model
The branding confusion with Grok is going to be tough. One needs a rename.
Nah, it's funnier this way.
@@undergroundxp What would be even funnier, and differentiating, is if they decided to tell everyone to pronounce this one "groque", like "baroque" 😅
Obligatory comment. Hope to keep seeing more videos from your channel 🙏
So there is a one off gain in putting the algorithm / transformer. In a perfect world that one off gain should be in the range of 50 to 100 x. But this costs a lot of money and is a big gamble. If that algorithm is tweaked then you need new chips. So it’s a balance between flexibility and the resulting risk and cost. All the major service providers and semi companies are looking at this. Groq have a time to market advantage but it’s not going to be that big. 6-12 months.
Big message. Short Nvidia stock.
I wonder if it is better with energy efficiency too?
Didn't explain how it works but 👍
nfi
I imagine NVidia will have an LPU inference only offering soon.
It's more likely they will buy Groq.
it cost the price of a new car
Good content but I want to know who I’m getting my info from. Are you an engineer?
I am a crypto shrill // AI hype boy.
Chamath is the Goat ❤
Does anyone know how to invest in this company
barely any difference between the gpt3.5 and groq respond times at 0:20 and 1:00
(Can you guys stop mentioning simtheory on podcast - I've been on waitlist for 4 months ?? - I hit the page each week in hot anticpation only to be disappointed things are still closed.... sighh......(fix this please))
Join the discord and send us a message with your email and we can let you in straight away: discord.gg/Rf7v6SZB
Groq was never explained. You just quoted the obvious time and time again. The title is just click bait.
Damn bro! Can see all the whites of your eyes! You on that high end addy or what?
Sanpaku eyes
How good is addy
Why would you have that annoying background music that makes impossible to listen to what you are saying? I don't get it...
To annoy you
i can relate to the difficulty of getting a wash for ones pig