Quick is an understatement. I wonder if we get to a point where the systems actually record our inputs before we press enter. E.g. they are working in the background as we type in our input and when we press enter it may be instantaneous ?
Current Groq configuration comes with limited stroage space of SRAM and it would not be sufficient for Training but Inference purpose. It 's comparable to Tesla's Dojo but with lesser computation power of Tesla Dojo. However when the potential buyers would have to ponder the limitation before making the purchase decision, Layers of KV Cache and Batch Size would make it only suitable for medium size computation. Not to mention that Complexity of Software should be taken into the consideration.
I was thinking about this also - maybe as an add-on PCIe card. For the crowd that may believe that like "not your keys, not your coins" - "not your cloud, not your data". I could see Apple as a candidate for buying them to make their already-fast Neural Engine on the motherboard into something that puts OpenAI out of business.
So there is a one off gain in putting the algorithm / transformer. In a perfect world that one off gain should be in the range of 50 to 100 x. But this costs a lot of money and is a big gamble. If that algorithm is tweaked then you need new chips. So it’s a balance between flexibility and the resulting risk and cost. All the major service providers and semi companies are looking at this. Groq have a time to market advantage but it’s not going to be that big. 6-12 months. Big message. Short Nvidia stock.
(Can you guys stop mentioning simtheory on podcast - I've been on waitlist for 4 months ?? - I hit the page each week in hot anticpation only to be disappointed things are still closed.... sighh......(fix this please))
Really good video. Looks very professional and the information was very easily understood. Great work!
Quick is an understatement. I wonder if we get to a point where the systems actually record our inputs before we press enter. E.g. they are working in the background as we type in our input and when we press enter it may be instantaneous ?
Current Groq configuration comes with limited stroage space of SRAM and it would not be sufficient for Training but Inference purpose. It 's comparable to Tesla's Dojo but with lesser computation power of Tesla Dojo. However when the potential buyers would have to ponder the limitation before making the purchase decision, Layers of KV Cache and Batch Size would make it only suitable for medium size computation. Not to mention that Complexity of Software should be taken into the consideration.
i feeel like this first chip is more a proof of concept or a marketing effort from Groq more than anything, this video is a perfect exapmle of that.
gotta make stuff for us hype boiz
Hopefully their next chip fixes these shortcomings.
Groq was before Grok
1:37 That "Silicon Wafer" looks suspiciously like a circle of corrugated cardboard covered in tinfoil...
It’s crazy though that it’s will cost about $2 million worth of cards to run a 70b model
Thanks for the video! So is Groq going to be sold as a chip sometime? Like will it be installed on a motherboard like a GPU or RAM or NVME?
I was thinking about this also - maybe as an add-on PCIe card. For the crowd that may believe that like "not your keys, not your coins" - "not your cloud, not your data". I could see Apple as a candidate for buying them to make their already-fast Neural Engine on the motherboard into something that puts OpenAI out of business.
Obligatory comment. Hope to keep seeing more videos from your channel 🙏
The branding confusion with Grok is going to be tough. One needs a rename.
Nah, it's funnier this way.
@@amykpop1 What would be even funnier, and differentiating, is if they decided to tell everyone to pronounce this one "groque", like "baroque" 😅
I wonder if it is better with energy efficiency too?
Does anyone know how to invest in this company
Groq was never explained. You just quoted the obvious time and time again. The title is just click bait.
So there is a one off gain in putting the algorithm / transformer. In a perfect world that one off gain should be in the range of 50 to 100 x. But this costs a lot of money and is a big gamble. If that algorithm is tweaked then you need new chips. So it’s a balance between flexibility and the resulting risk and cost. All the major service providers and semi companies are looking at this. Groq have a time to market advantage but it’s not going to be that big. 6-12 months.
Big message. Short Nvidia stock.
I imagine NVidia will have an LPU inference only offering soon.
It's more likely they will buy Groq.
Didn't explain how it works but 👍
nfi
barely any difference between the gpt3.5 and groq respond times at 0:20 and 1:00
it cost the price of a new car
Chamath is the Goat ❤
Good content but I want to know who I’m getting my info from. Are you an engineer?
I am a crypto shrill // AI hype boy.
(Can you guys stop mentioning simtheory on podcast - I've been on waitlist for 4 months ?? - I hit the page each week in hot anticpation only to be disappointed things are still closed.... sighh......(fix this please))
Join the discord and send us a message with your email and we can let you in straight away: discord.gg/Rf7v6SZB
Damn bro! Can see all the whites of your eyes! You on that high end addy or what?
Sanpaku eyes
How good is addy
Why would you have that annoying background music that makes impossible to listen to what you are saying? I don't get it...
To annoy you