Check out HubSpot's ChatGPT at work bundle! clickhubspot.com/twc and I missed out a paper that combined mamba & attention, you can find it here: arxiv.org/abs/2402.04248 what an interesting timeline this is
Somebody (I nominate you) needs to chart knowledge half-life and relate to the landfill and SciFi and "human uselessness". Somehow this deserves at least multiple theses. If there was time.... I think that would be more telling than Kurzweil's Exponential Tech Acceleration chart. "The Matrix is a place to hide and play games...."
FYI: 7T is about the entire infrastructure not just chip factories and the fact that OpenAI doesn't own much compute in comparison to Meta, Microsoft, Nvidia, or Microsoft.
@@itzhexen0 You absolutely should be, it won't be about keeping you safe but them, and if they are safe, well a nation of sheep will beget a Government of wolves.
@@jonathanberry1111 Well there are a lot of nutty people in society and I think it's needed. It probably won't be about keeping you safe either. But what else would they need 7 trillion dollars for? Chips for themselves? Because they won't be passing them out to everyone in the USA.
That anime edit was one of the sickest media pieces I've seen, but unfortunatelly I have no friends in the intersection of jujutsu enjoyers and ai reaserch conisseurs, who would appreciate it wholly
I love seeing this trend of LLMs getting quicker and using less resources. I think we are only a few breakthroughs away from a point where LLMs can begin running on mobile devices locally at reasonable speeds. Right now companies are spending major resources on making the models smarter through the models themselves. However, make the model small and quick enough, and you could run it multiple times, prompted by hard-coded logic, to possibly accomplish the same things as the larger models without the need for as much power or space (at the cost of time). This could allow an LLM to exist on robots without being connected to a service. The technology is in the works for quick instruction following for robots, so an LLM being able to feed the robot instructions makes the robot self guiding, which would be a sight.
I guess ² is still an exponent by which we're scaling here. I'm sure anyone watching this video will know what was meant. A correction should probably still be made.
@@XashA12Musk the manga is Jujutsu kaisen. He took from multiple parts. You can check chapter 75 for the "throughout the heavens and the earth" and chapter 221 for the "nah, I'd win"
Yes but how much would it cost to port something like GPT4 to Mamba or if they even can or they'd have to start from scratch? It's probably wont be the only architecture to come out so i imagine OpenAI are waiting for something that is very clearly way better in almost all categories compared to transformers
They'd probably have to start from scratch. However it could take much less time to train. If it takes 1/1000 of the time to train with similar real world performance, it might become worth it. Transformers are proven while novel architectures like mamba are not. OpenAI is selling chatgpt after all so it may not be worth it for them.
My greatest hope is that is can really run 6B models like they're 2B, was that for train or for actually running them? If it's for running them, then even the 40B param issue won't matter for local models, most consumer computers would gladly take 40B models that run like they're 20s
awesome breakdown. When the other AI hype channels asked bycloud if he could go head to head with their surface level analysis, bycloud responded "Nah, I'd win" (DEEPFRIED BASS)
I'm not sure if everyone reading this knows, though with cloud's audience it's probably most. Going from something exponential to something linear is a GODSEND. The title says 1000x, take that as you will but even if it's just 10x, due to how exponentials work this would still save much way more than 1000x in the future because if it exponentially increased the computational costs would go way out of control, but with linear it's way WAY more manageable. If this is real, it will be a complete game changer.
don't get your panties wet. It might just be smoke until tested to see if it works with extremely large models. But the thing about it understanding vision better, can't they just codify data into visuals and then have the LLM train on that, then you build another LLM that can translate the input and output and the problem is solved? But then again, even with the vision based MAMBA, large models still haven't been tested so who knows.
what if there is a 70B mamba ? it can surpass existing ones? I don't saw in any place comparations where they compare mamba 70b with any other big model. perhaps it would be a decisive analysis to see how it performes
I assume that bigger models are still in the works. Most attention is on transformers-based models, so the money and resources to train a 70b model for Mamba are taking longer to gather. I'm definitely looking forward to seeing what becomes of it though.
I come back to this video regularly just because of that sick JJK edit! Oh and also because it has the perfect balance between technical explanation and entertainment.
@@Hollowed2wiz what is interesting is that gpt4 can spell this word one letter at time if asked and then give right answer about letter N count. So it seems that despite tokinezation gpt4 knows something about spelling...
If you were one of the professors at my school I would never miss a class lol. You are great at breaking down concepts and making it funny. Keep up the good work!
These comments make me feel stupid 😭 All I understood is that mamba is faster than traditional transformer thing (the party explanation was awesome, thank you), and that mamba abandons the tokenization and uses... something else instead... I kinda wish you would summarize the video at the end for silly creatures like me But for now it's time to rewatch everything!! 🥳
I do love Mamba for faster token/sec. But there's still a long road to make it able to extract key information from long text. For now it still feels like Bart or Gemma 2b for short prompt
Great video, I already heard about Mamba, but didn’t get into it myself! The lobotomy Kaisen edit went really hard haha. Any chance you will be making a video about Liquid Neural Networks? Keep up the good work :)
I wonder if OpenAI switches to Mamba architecture if they will drop the "GPT" branding since technically the T will not apply anymore. I wonder if "GPT" will be like how Boomers would call every game console a "nintendo" and just used by the mainstream to mean every LLM, no matter the underlying architecture.
Even if this doesn't replace transformers, this looks like a very promising way to replace tokenization/word vectors by having a layer read the bytes and output vector tokens
It's not either-or. An attention layer has many heads, and for sure some of these learn tasks that mamba is better at. So, replace some attention heads with mambas and find out which ratio is best.
tldr by Perplexity: The video titled "Mamba Might Just Make LLMs 1000x Cheaper..." discusses a new AI architecture known as Mamba, which aims to significantly reduce the cost and improve the efficiency of Large Language Models (LLMs). Mamba differentiates itself from traditional Transformer models by utilizing a State Space Sequence Model (S4) and a selective mechanism, which allows for linear scaling and faster inference times. This architecture shows promise not only in language tasks but also in vision tasks, indicating its potential as a versatile tool in AI development. Mamba's approach to learning directly from raw byte patterns rather than tokenized text addresses the challenges posed by tokenization, such as distortion in text representation. This method enables Mamba to generate more accurate and coherent text, especially for long or complex sequences. Despite its advantages, Mamba faces challenges like information loss in long contexts. However, its introduction represents a significant advancement in AI architectures, potentially challenging the dominance of Transformers.
I imagine Transformer guys are seeing MAMBA as a big 2 way transformer and yank it in the Transformer architecture forming multi-architecture transformer model.
the video was good and i liked your style, but then the JJK edit dropped actual fire 🔥 i hope you don't mind me reposting the JJK edit section of your video tell me if you want me to take it down
7:25mins in woah!!! I want all of my tech news delivered in this format!! excitingly eccentric, suspense filled, noir comic animations with deep, rich & sexy actor voice overs.. so cool! @bycloudAI
00:27 🔍 Self-attention mechanism in Transformers enables advanced text completion but struggles with basic arithmetic; companies integrate calculators to mitigate issues. 02:29 🔄 Mamba, a new model, addresses Transformer inefficiencies by scaling linearly, not exponentially, and doesn't rely on the attention mechanism. 05:32 🚀 Mamba potentially offers 1000x cheaper scaling than GPT-4, with quadratic scaling improvement and faster calculations, revolutionizing AI chatbots. 07:10 🧬 Mamba's long sequence handling benefits DNA modeling, audio synthesis, and analyzing high-resolution images or long-form videos. 10:29 💻 Mamba Bite model learns directly from raw bytes, eliminating tokenization biases, enhancing long sequence comprehension, and potentially enabling true multimodal models. 12:48 ⚠ Potential downside of Mamba: "Lost in the Middle" issue may lead to permanent loss of information in very long contexts; further research needed to address this.
People are sleeping on vanilla transformers. Trying to replace it without even fully optimizing a transformer. Fyi they can have sublinear memory consumption with clever stretching of the kv cache. Removing the need of an exact kv cache. Sleeping 😂.
![Breaking Down Meta's Billion Dollar LLM Blueprint [Llama-3.1 Full Breakdown]](http://i.ytimg.com/vi/68_BKkymYs4/mqdefault.jpg)
Check out HubSpot's ChatGPT at work bundle! clickhubspot.com/twc
and I missed out a paper that combined mamba & attention, you can find it here: arxiv.org/abs/2402.04248
what an interesting timeline this is
JJK edit after joe mama joke. This video is a masterpiece.
Somebody (I nominate you) needs to chart knowledge half-life and relate to the landfill and SciFi and "human uselessness". Somehow this deserves at least multiple theses. If there was time.... I think that would be more telling than Kurzweil's Exponential Tech Acceleration chart. "The Matrix is a place to hide and play games...."
Can you share the video assets for the anime edit? I wanna try using it in the future. Also how did you get the voice?
mark your vids where the examples are. i for example always skip straight to examples when im learning
Bro dropped the hardest LLM anime edit and thought we wouldnt notice
7:24 if you want to experience AI lobotomy
Wasn't expecting that AT ALL 🤣🤣🤣
This why he got a new sub 😂
wouldn't*
Bruv I know. I made a clip and immediately shared. I almost feel like all technical information should be conveyed this way.
7:20 the greatest LLM anime of all time begins(JJK is within 2 letters of LLM)
and by lack of competition...sadly the worst as well
LuLutsu Maisen
Good so I guess OpenAI no longer needs 7 Trillion dollars for chip factories.
😂
FYI: 7T is about the entire infrastructure not just chip factories and the fact that OpenAI doesn't own much compute in comparison to Meta, Microsoft, Nvidia, or Microsoft.
@sco444 Yeah mass surveillance infrastructure. Not that I'm against that.
@@itzhexen0 You absolutely should be, it won't be about keeping you safe but them, and if they are safe, well a nation of sheep will beget a Government of wolves.
@@jonathanberry1111 Well there are a lot of nutty people in society and I think it's needed. It probably won't be about keeping you safe either. But what else would they need 7 trillion dollars for? Chips for themselves? Because they won't be passing them out to everyone in the USA.
That anime edit was one of the sickest media pieces I've seen, but unfortunatelly I have no friends in the intersection of jujutsu enjoyers and ai reaserch conisseurs, who would appreciate it wholly
Yeah, when I saw it I was like: I need to show it to... Wait who will ever understand it among my friends? No one
But U have Us in comments section so we can laugh together 😂
😂 I did not expect The JJK EDIT and died laughing
Whats JJK??
Jujutsu kaisen or smth like that@@Itachi_Uchia1
@@Itachi_Uchia1 Jujutsu Kaisen
Yes 🤣 So on point
I love seeing this trend of LLMs getting quicker and using less resources. I think we are only a few breakthroughs away from a point where LLMs can begin running on mobile devices locally at reasonable speeds. Right now companies are spending major resources on making the models smarter through the models themselves. However, make the model small and quick enough, and you could run it multiple times, prompted by hard-coded logic, to possibly accomplish the same things as the larger models without the need for as much power or space (at the cost of time). This could allow an LLM to exist on robots without being connected to a service. The technology is in the works for quick instruction following for robots, so an LLM being able to feed the robot instructions makes the robot self guiding, which would be a sight.
In the early 2000s, here in Russia, Mamba was a very popular dating site. Good to hear they are now at the frontier of AI development!!!
Just exactly like how RUclips used to be a dating site too! The story repeats itself.
also 'cope' (the word in the thumb) is 2019-ish 4chan troll word; this video is nostalgic in many aspects!
@@16876 isn't it ironic how Twitter started abusing the FUCK out of 4chan lingo
@@16876 No one asked nerd
didn’t expect lobotomy kaisen to make its way to the LLM and AI space😭😭 best thing ever
No one:
Mamaba: Nah! I'd win.
"Exponentially" should stop being misused for everything that is bigger than linear... Quadratic != exponential
what is x^3?
It isn't wrong to call a square a rectangle.
@@WoolyCow Quadratic is polynomial, Exponential is exponential.
@@nanubalagnanasai3006 yeah mb, i must be stupid lol
@@losttale1cubic
naaah lobotomy kaisen is taking over everything i swear 💀💀💀💀
Just for a few months than all the lobotomies will forgor
Just for a few months than all the lobotomies will forgor
it scales quadratically not exponentially
oops i meant it metaphorically
that was a bad word choice lol
I guess ² is still an exponent by which we're scaling here. I'm sure anyone watching this video will know what was meant. A correction should probably still be made.
@@Alice_Fumo But exponential means that it scales according to the nth power. n² is polynomial, better than linear but not exponential like 2ⁿ.
Came here to say this. Quadratic and exponential is a huge difference.
@@LeoVital *worse than linear
"Stand proud Transformer, you are strong."
- Mamba
throughout youtube clickbait and interesting facts you are the honored one
The last thing I expected was a jjk edit
The Gojo reference made me shout out loud like a little fangirl :')
can you give me the original source of that anime edit ?
@@XashA12Musk the manga is Jujutsu kaisen. He took from multiple parts. You can check chapter 75 for the "throughout the heavens and the earth" and chapter 221 for the "nah, I'd win"
this is crazy, everytime i click i think im going to watch a fireship vid
bait
Yes but how much would it cost to port something like GPT4 to Mamba or if they even can or they'd have to start from scratch?
It's probably wont be the only architecture to come out so i imagine OpenAI are waiting for something that is very clearly way better in almost all categories compared to transformers
They'd probably have to start from scratch. However it could take much less time to train. If it takes 1/1000 of the time to train with similar real world performance, it might become worth it. Transformers are proven while novel architectures like mamba are not. OpenAI is selling chatgpt after all so it may not be worth it for them.
i have some hope for byte mamba but the architecture has drawbacks and seems more like an intermediary step before something greater that builds on it
My greatest hope is that is can really run 6B models like they're 2B, was that for train or for actually running them?
If it's for running them, then even the 40B param issue won't matter for local models, most consumer computers would gladly take 40B models that run like they're 20s
awesome breakdown. When the other AI hype channels asked bycloud if he could go head to head with their surface level analysis, bycloud responded "Nah, I'd win" (DEEPFRIED BASS)
I'm not sure if everyone reading this knows, though with cloud's audience it's probably most.
Going from something exponential to something linear is a GODSEND.
The title says 1000x, take that as you will but even if it's just 10x, due to how exponentials work this would still save much way more than 1000x in the future because if it exponentially increased the computational costs would go way out of control, but with linear it's way WAY more manageable.
If this is real, it will be a complete game changer.
Technically, LLM context length increases quadratically, not exponentially.
don't get your panties wet. It might just be smoke until tested to see if it works with extremely large models.
But the thing about it understanding vision better, can't they just codify data into visuals and then have the LLM train on that, then you build another LLM that can translate the input and output and the problem is solved?
But then again, even with the vision based MAMBA, large models still haven't been tested so who knows.
@@verigumetin4291the thing with that method is that yes it's possible but problem is similar to moe
I was not expecting to get a lobotomy while watching an LLM news video today...
Ngl the thumbnail tricked me, thought it was a fireship video, but it worked lol and I’m still watching.
same!!
this was the most entertaining and somehow equally educational llm/ai video i’ve ever watched
I just want to see a 7B mamba model trained on the same data as a 7B transformer model and get to try them both and test them on certain abilities.
many of us want it too
what if there is a 70B mamba ? it can surpass existing ones? I don't saw in any place comparations where they compare mamba 70b with any other big model. perhaps it would be a decisive analysis to see how it performes
I assume that bigger models are still in the works. Most attention is on transformers-based models, so the money and resources to train a 70b model for Mamba are taking longer to gather. I'm definitely looking forward to seeing what becomes of it though.
Dude that was such a sick anime cut in clip. How did you make that? D that script? All ai?
I come back to this video regularly just because of that sick JJK edit! Oh and also because it has the perfect balance between technical explanation and entertainment.
Ngl, my bro is the Jay-Z of LLM education...out here dropping bangers.
Woooooowwww, 8ish minutes in was a mic drop I didn't expect. First time here, not the last.
Dangit, now I have Lou Bega's Mambo No. 5 stuck in my head!
that was a great part at 7:21
You gained a like due to the high-quality of your video, however, when the JJK edit dropped, you gained my subscription and my worship. 🛐
I just wish to be this talented- amazing video
Just wait until they discover Mamba No.5. There will be no going back...
Thanks for the papers in the description, I just put them a urls=[] and hydrated my s3 and vdb with them 5:42 😎
This video is where I will start with my thesis.
I have no idea how LLMs work and I still understood some of it and the implications. Which is to say, this is a great video my dude.
I alone am the subquadratic one.
Such a great video, and FUNNY! Glad you're making these
5:39 It's only 1000 times cheaper because the price was per 1000 tokens. If it was per 4000 tokens, it would get 4000 times cheaper, and so on. 😊
finally understood why AI cannot tell how many letters N word banana contains )
damn, I just tested it with gpt4, and it said that there are 3 n in banana.
It's so funny 🤣
@@Hollowed2wiz what is interesting is that gpt4 can spell this word one letter at time if asked and then give right answer about letter N count. So it seems that despite tokinezation gpt4 knows something about spelling...
Straight up stealing fireship viewers with these thumbnails ☠️
I completely agree but at least the context doesn't let down
If you were one of the professors at my school I would never miss a class lol. You are great at breaking down concepts and making it funny. Keep up the good work!
yeah, I understand you mamba, I too have a "lost in the middle" problem
Your videos are just the best. Humor and knowledge in its best combination
Sooner or later we would need a bytes-level model architecture for multi-modality. Hope the test result for this one be good🙏 .
Cool video btw 👌
These comments make me feel stupid 😭
All I understood is that mamba is faster than traditional transformer thing (the party explanation was awesome, thank you), and that mamba abandons the tokenization and uses... something else instead... I kinda wish you would summarize the video at the end for silly creatures like me
But for now it's time to rewatch everything!! 🥳
The edit made me understand than without :D
Hardest AI channel 🪨 🤘
subbed...
7:22 Fire 🔥🔥🔥🔥🔥
Without a serious upscaled mamba it will be going no were expect for niche areas
13:15 that’s the most passive paper name I’ve ever seen
I do love Mamba for faster token/sec. But there's still a long road to make it able to extract key information from long text. For now it still feels like Bart or Gemma 2b for short prompt
Wonder what Google are doing with 1M - 10M+ context Gemini Pro 1.5?
Wtf did I just watch? This is like the best ever.
VMamba has updated the scaling chart at 9:35. Performance keeps increasing with increased model size, but not much
I cannot explain how much I enjoyed that edit
Great video, I already heard about Mamba, but didn’t get into it myself! The lobotomy Kaisen edit went really hard haha.
Any chance you will be making a video about Liquid Neural Networks?
Keep up the good work :)
As an electrical engineer: REAL TRANSFORMERS HAVE WINDINGS.
I've been attending parties with the mamba method forever. I may have been doing that wrong.
Plz tell me how do you select gifs for videos it's so precised that I doubt it's made by chatgpt no human
I wonder if OpenAI switches to Mamba architecture if they will drop the "GPT" branding since technically the T will not apply anymore. I wonder if "GPT" will be like how Boomers would call every game console a "nintendo" and just used by the mainstream to mean every LLM, no matter the underlying architecture.
Lol Nintendo, but doesn't GPT = General Pre-Trained, not "-Transformer"?
Even if this doesn't replace transformers, this looks like a very promising way to replace tokenization/word vectors by having a layer read the bytes and output vector tokens
Bro. You make these LLM videos so interesting and funny. How do you come up with these? Keep it up
It's not either-or. An attention layer has many heads, and for sure some of these learn tasks that mamba is better at. So, replace some attention heads with mambas and find out which ratio is best.
Nice ! I wonder if this would be compatible with Groq's chip.
This and the LPU that Groq uses are going to be insane together.
That thing was still created purely for a Transformers style model.
They will need to make a new arch
5:02 i am all in for non-sound memes. at least it doesnt make me a weirdo when watching without earphones in an open space.
tldr by Perplexity:
The video titled "Mamba Might Just Make LLMs 1000x Cheaper..." discusses a new AI architecture known as Mamba, which aims to significantly reduce the cost and improve the efficiency of Large Language Models (LLMs). Mamba differentiates itself from traditional Transformer models by utilizing a State Space Sequence Model (S4) and a selective mechanism, which allows for linear scaling and faster inference times. This architecture shows promise not only in language tasks but also in vision tasks, indicating its potential as a versatile tool in AI development. Mamba's approach to learning directly from raw byte patterns rather than tokenized text addresses the challenges posed by tokenization, such as distortion in text representation. This method enables Mamba to generate more accurate and coherent text, especially for long or complex sequences. Despite its advantages, Mamba faces challenges like information loss in long contexts. However, its introduction represents a significant advancement in AI architectures, potentially challenging the dominance of Transformers.
the JJK edit was INSANELY funny!
This whole video is Gold.
The LLM anime edit earned my subscription
bro the content on this channel is for such a niche audience only 5 people on RUclips will get all the memes and understand the science
I imagine Transformer guys are seeing MAMBA as a big 2 way transformer and yank it in the Transformer architecture forming multi-architecture transformer model.
the video was good and i liked your style, but then the JJK edit dropped
actual fire 🔥
i hope you don't mind me reposting the JJK edit section of your video
tell me if you want me to take it down
Didn't expect to see some jojo and gojo references in a AI model video, awesome!
Bro the JJK edit is superb
Cool reference to the router edit
Lobotomy kaisen edit was PEAK
7:25mins in woah!!! I want all of my tech news delivered in this format!! excitingly eccentric, suspense filled, noir comic animations with deep, rich & sexy actor voice overs.. so cool! @bycloudAI
actually the start of the video is incorrect since bard was merged with gemini (though this video probably started production before that)
mamba is snake in luganda
Yoo that LLM'S Anime edit hit hardddd😂😂
11:45 why can't we force it to process numbers as sequences of digits in certain contexts?
I expected "Are your hidden states linear because you're a State Space Model? Because I can't seem to figure out your next move."
:D
But Bycloud, what about Ring Attention? Doesn't it allow for near infinite context too?
the jjk edit killed me, bro i love you
The JJK bit was great
what about ring and flash attention? dont gemini or sora uses different atfention than mh attention to get 7M tokens?
Fantastic sub. Need more neutral analysis of cutting edge
I would not have commented if it wasn't for that "nah I'd win" Mamba edit
Dude... We are dangerously close to a bearish breakout.
00:27 🔍 Self-attention mechanism in Transformers enables advanced text completion but struggles with basic arithmetic; companies integrate calculators to mitigate issues.
02:29 🔄 Mamba, a new model, addresses Transformer inefficiencies by scaling linearly, not exponentially, and doesn't rely on the attention mechanism.
05:32 🚀 Mamba potentially offers 1000x cheaper scaling than GPT-4, with quadratic scaling improvement and faster calculations, revolutionizing AI chatbots.
07:10 🧬 Mamba's long sequence handling benefits DNA modeling, audio synthesis, and analyzing high-resolution images or long-form videos.
10:29 💻 Mamba Bite model learns directly from raw bytes, eliminating tokenization biases, enhancing long sequence comprehension, and potentially enabling true multimodal models.
12:48 ⚠ Potential downside of Mamba: "Lost in the Middle" issue may lead to permanent loss of information in very long contexts; further research needed to address this.
GOD LEVEL EDIT, WHO IS THIS RUclipsR?
People are sleeping on vanilla transformers. Trying to replace it without even fully optimizing a transformer. Fyi they can have sublinear memory consumption with clever stretching of the kv cache. Removing the need of an exact kv cache. Sleeping 😂.
the LLM anime edit was fucking gold
bro that anime thing was just incredible
that gojo edit has to be the hardest thing ive seen in years wtf were you on when that came to your mind