Check out HubSpot's ChatGPT at work bundle! clickhubspot.com/twc and I missed out a paper that combined mamba & attention, you can find it here: arxiv.org/abs/2402.04248 what an interesting timeline this is
Somebody (I nominate you) needs to chart knowledge half-life and relate to the landfill and SciFi and "human uselessness". Somehow this deserves at least multiple theses. If there was time.... I think that would be more telling than Kurzweil's Exponential Tech Acceleration chart. "The Matrix is a place to hide and play games...."
That anime edit was one of the sickest media pieces I've seen, but unfortunatelly I have no friends in the intersection of jujutsu enjoyers and ai reaserch conisseurs, who would appreciate it wholly
I love seeing this trend of LLMs getting quicker and using less resources. I think we are only a few breakthroughs away from a point where LLMs can begin running on mobile devices locally at reasonable speeds. Right now companies are spending major resources on making the models smarter through the models themselves. However, make the model small and quick enough, and you could run it multiple times, prompted by hard-coded logic, to possibly accomplish the same things as the larger models without the need for as much power or space (at the cost of time). This could allow an LLM to exist on robots without being connected to a service. The technology is in the works for quick instruction following for robots, so an LLM being able to feed the robot instructions makes the robot self guiding, which would be a sight.
@@XashA12Musk the manga is Jujutsu kaisen. He took from multiple parts. You can check chapter 75 for the "throughout the heavens and the earth" and chapter 221 for the "nah, I'd win"
I guess ² is still an exponent by which we're scaling here. I'm sure anyone watching this video will know what was meant. A correction should probably still be made.
Yes but how much would it cost to port something like GPT4 to Mamba or if they even can or they'd have to start from scratch? It's probably wont be the only architecture to come out so i imagine OpenAI are waiting for something that is very clearly way better in almost all categories compared to transformers
They'd probably have to start from scratch. However it could take much less time to train. If it takes 1/1000 of the time to train with similar real world performance, it might become worth it. Transformers are proven while novel architectures like mamba are not. OpenAI is selling chatgpt after all so it may not be worth it for them.
My greatest hope is that is can really run 6B models like they're 2B, was that for train or for actually running them? If it's for running them, then even the 40B param issue won't matter for local models, most consumer computers would gladly take 40B models that run like they're 20s
awesome breakdown. When the other AI hype channels asked bycloud if he could go head to head with their surface level analysis, bycloud responded "Nah, I'd win" (DEEPFRIED BASS)
I'm not sure if everyone reading this knows, though with cloud's audience it's probably most. Going from something exponential to something linear is a GODSEND. The title says 1000x, take that as you will but even if it's just 10x, due to how exponentials work this would still save much way more than 1000x in the future because if it exponentially increased the computational costs would go way out of control, but with linear it's way WAY more manageable. If this is real, it will be a complete game changer.
don't get your panties wet. It might just be smoke until tested to see if it works with extremely large models. But the thing about it understanding vision better, can't they just codify data into visuals and then have the LLM train on that, then you build another LLM that can translate the input and output and the problem is solved? But then again, even with the vision based MAMBA, large models still haven't been tested so who knows.
I come back to this video regularly just because of that sick JJK edit! Oh and also because it has the perfect balance between technical explanation and entertainment.
00:27 🔍 Self-attention mechanism in Transformers enables advanced text completion but struggles with basic arithmetic; companies integrate calculators to mitigate issues. 02:29 🔄 Mamba, a new model, addresses Transformer inefficiencies by scaling linearly, not exponentially, and doesn't rely on the attention mechanism. 05:32 🚀 Mamba potentially offers 1000x cheaper scaling than GPT-4, with quadratic scaling improvement and faster calculations, revolutionizing AI chatbots. 07:10 🧬 Mamba's long sequence handling benefits DNA modeling, audio synthesis, and analyzing high-resolution images or long-form videos. 10:29 💻 Mamba Bite model learns directly from raw bytes, eliminating tokenization biases, enhancing long sequence comprehension, and potentially enabling true multimodal models. 12:48 ⚠ Potential downside of Mamba: "Lost in the Middle" issue may lead to permanent loss of information in very long contexts; further research needed to address this.
what if there is a 70B mamba ? it can surpass existing ones? I don't saw in any place comparations where they compare mamba 70b with any other big model. perhaps it would be a decisive analysis to see how it performes
I assume that bigger models are still in the works. Most attention is on transformers-based models, so the money and resources to train a 70b model for Mamba are taking longer to gather. I'm definitely looking forward to seeing what becomes of it though.
If you were one of the professors at my school I would never miss a class lol. You are great at breaking down concepts and making it funny. Keep up the good work!
7:25mins in woah!!! I want all of my tech news delivered in this format!! excitingly eccentric, suspense filled, noir comic animations with deep, rich & sexy actor voice overs.. so cool! @bycloudAI
@@Hollowed2wiz what is interesting is that gpt4 can spell this word one letter at time if asked and then give right answer about letter N count. So it seems that despite tokinezation gpt4 knows something about spelling...
These comments make me feel stupid 😭 All I understood is that mamba is faster than traditional transformer thing (the party explanation was awesome, thank you), and that mamba abandons the tokenization and uses... something else instead... I kinda wish you would summarize the video at the end for silly creatures like me But for now it's time to rewatch everything!! 🥳
I wonder if OpenAI switches to Mamba architecture if they will drop the "GPT" branding since technically the T will not apply anymore. I wonder if "GPT" will be like how Boomers would call every game console a "nintendo" and just used by the mainstream to mean every LLM, no matter the underlying architecture.
tldr by Perplexity: The video titled "Mamba Might Just Make LLMs 1000x Cheaper..." discusses a new AI architecture known as Mamba, which aims to significantly reduce the cost and improve the efficiency of Large Language Models (LLMs). Mamba differentiates itself from traditional Transformer models by utilizing a State Space Sequence Model (S4) and a selective mechanism, which allows for linear scaling and faster inference times. This architecture shows promise not only in language tasks but also in vision tasks, indicating its potential as a versatile tool in AI development. Mamba's approach to learning directly from raw byte patterns rather than tokenized text addresses the challenges posed by tokenization, such as distortion in text representation. This method enables Mamba to generate more accurate and coherent text, especially for long or complex sequences. Despite its advantages, Mamba faces challenges like information loss in long contexts. However, its introduction represents a significant advancement in AI architectures, potentially challenging the dominance of Transformers.
Even if this doesn't replace transformers, this looks like a very promising way to replace tokenization/word vectors by having a layer read the bytes and output vector tokens
Great video, I already heard about Mamba, but didn’t get into it myself! The lobotomy Kaisen edit went really hard haha. Any chance you will be making a video about Liquid Neural Networks? Keep up the good work :)
I do love Mamba for faster token/sec. But there's still a long road to make it able to extract key information from long text. For now it still feels like Bart or Gemma 2b for short prompt
I imagine Transformer guys are seeing MAMBA as a big 2 way transformer and yank it in the Transformer architecture forming multi-architecture transformer model.
the video was good and i liked your style, but then the JJK edit dropped actual fire 🔥 i hope you don't mind me reposting the JJK edit section of your video tell me if you want me to take it down
![BLACK BAG - Official Trailer [HD] - Only in Theaters March 14](http://i.ytimg.com/vi/Du0Xp8WX_7I/mqdefault.jpg)
Check out HubSpot's ChatGPT at work bundle! clickhubspot.com/twc
and I missed out a paper that combined mamba & attention, you can find it here: arxiv.org/abs/2402.04248
what an interesting timeline this is
JJK edit after joe mama joke. This video is a masterpiece.
Somebody (I nominate you) needs to chart knowledge half-life and relate to the landfill and SciFi and "human uselessness". Somehow this deserves at least multiple theses. If there was time.... I think that would be more telling than Kurzweil's Exponential Tech Acceleration chart. "The Matrix is a place to hide and play games...."
Can you share the video assets for the anime edit? I wanna try using it in the future. Also how did you get the voice?
mark your vids where the examples are. i for example always skip straight to examples when im learning
Bro dropped the hardest LLM anime edit and thought we wouldnt notice
7:24 if you want to experience AI lobotomy
Wasn't expecting that AT ALL 🤣🤣🤣
This why he got a new sub 😂
wouldn't*
Bruv I know. I made a clip and immediately shared. I almost feel like all technical information should be conveyed this way.
😂 I did not expect The JJK EDIT and died laughing
Whats JJK??
Jujutsu kaisen or smth like that@@Itachi_Uchia1
@@Itachi_Uchia1 Jujutsu Kaisen
Yes 🤣 So on point
7:20 the greatest LLM anime of all time begins(JJK is within 2 letters of LLM)
and by lack of competition...sadly the worst as well
LuLutsu Maisen
@@revimfadli4666
That anime edit was one of the sickest media pieces I've seen, but unfortunatelly I have no friends in the intersection of jujutsu enjoyers and ai reaserch conisseurs, who would appreciate it wholly
Yeah, when I saw it I was like: I need to show it to... Wait who will ever understand it among my friends? No one
But U have Us in comments section so we can laugh together 😂
No one:
Mamaba: Nah! I'd win.
"Exponentially" should stop being misused for everything that is bigger than linear... Quadratic != exponential
what is x^3?
It isn't wrong to call a square a rectangle.
@@WoolyCow Quadratic is polynomial, Exponential is exponential.
@@nanubalagnanasai3006 yeah mb, i must be stupid lol
@@losttale1cubic
"Stand proud Transformer, you are strong."
- Mamba
In the early 2000s, here in Russia, Mamba was a very popular dating site. Good to hear they are now at the frontier of AI development!!!
Just exactly like how RUclips used to be a dating site too! The story repeats itself.
also 'cope' (the word in the thumb) is 2019-ish 4chan troll word; this video is nostalgic in many aspects!
@@16876 isn't it ironic how Twitter started abusing the FUCK out of 4chan lingo
@@16876 No one asked nerd
didn’t expect lobotomy kaisen to make its way to the LLM and AI space😭😭 best thing ever
I love seeing this trend of LLMs getting quicker and using less resources. I think we are only a few breakthroughs away from a point where LLMs can begin running on mobile devices locally at reasonable speeds. Right now companies are spending major resources on making the models smarter through the models themselves. However, make the model small and quick enough, and you could run it multiple times, prompted by hard-coded logic, to possibly accomplish the same things as the larger models without the need for as much power or space (at the cost of time). This could allow an LLM to exist on robots without being connected to a service. The technology is in the works for quick instruction following for robots, so an LLM being able to feed the robot instructions makes the robot self guiding, which would be a sight.
throughout youtube clickbait and interesting facts you are the honored one
this is crazy, everytime i click i think im going to watch a fireship vid
bait
The Gojo reference made me shout out loud like a little fangirl :')
can you give me the original source of that anime edit ?
@@XashA12Musk the manga is Jujutsu kaisen. He took from multiple parts. You can check chapter 75 for the "throughout the heavens and the earth" and chapter 221 for the "nah, I'd win"
it scales quadratically not exponentially
oops i meant it metaphorically
that was a bad word choice lol
I guess ² is still an exponent by which we're scaling here. I'm sure anyone watching this video will know what was meant. A correction should probably still be made.
@@Alice_Fumo But exponential means that it scales according to the nth power. n² is polynomial, better than linear but not exponential like 2ⁿ.
Came here to say this. Quadratic and exponential is a huge difference.
@@LeoVital *worse than linear
naaah lobotomy kaisen is taking over everything i swear 💀💀💀💀
Just for a few months than all the lobotomies will forgor
Just for a few months than all the lobotomies will forgor
The last thing I expected was a jjk edit
this was the most entertaining and somehow equally educational llm/ai video i’ve ever watched
Ngl the thumbnail tricked me, thought it was a fireship video, but it worked lol and I’m still watching.
same!!
Yes but how much would it cost to port something like GPT4 to Mamba or if they even can or they'd have to start from scratch?
It's probably wont be the only architecture to come out so i imagine OpenAI are waiting for something that is very clearly way better in almost all categories compared to transformers
They'd probably have to start from scratch. However it could take much less time to train. If it takes 1/1000 of the time to train with similar real world performance, it might become worth it. Transformers are proven while novel architectures like mamba are not. OpenAI is selling chatgpt after all so it may not be worth it for them.
i have some hope for byte mamba but the architecture has drawbacks and seems more like an intermediary step before something greater that builds on it
My greatest hope is that is can really run 6B models like they're 2B, was that for train or for actually running them?
If it's for running them, then even the 40B param issue won't matter for local models, most consumer computers would gladly take 40B models that run like they're 20s
I was not expecting to get a lobotomy while watching an LLM news video today...
awesome breakdown. When the other AI hype channels asked bycloud if he could go head to head with their surface level analysis, bycloud responded "Nah, I'd win" (DEEPFRIED BASS)
I'm not sure if everyone reading this knows, though with cloud's audience it's probably most.
Going from something exponential to something linear is a GODSEND.
The title says 1000x, take that as you will but even if it's just 10x, due to how exponentials work this would still save much way more than 1000x in the future because if it exponentially increased the computational costs would go way out of control, but with linear it's way WAY more manageable.
If this is real, it will be a complete game changer.
Technically, LLM context length increases quadratically, not exponentially.
don't get your panties wet. It might just be smoke until tested to see if it works with extremely large models.
But the thing about it understanding vision better, can't they just codify data into visuals and then have the LLM train on that, then you build another LLM that can translate the input and output and the problem is solved?
But then again, even with the vision based MAMBA, large models still haven't been tested so who knows.
@@verigumetin4291the thing with that method is that yes it's possible but problem is similar to moe
Dude that was such a sick anime cut in clip. How did you make that? D that script? All ai?
I come back to this video regularly just because of that sick JJK edit! Oh and also because it has the perfect balance between technical explanation and entertainment.
You gained a like due to the high-quality of your video, however, when the JJK edit dropped, you gained my subscription and my worship. 🛐
Straight up stealing fireship viewers with these thumbnails ☠️
I completely agree but at least the context doesn't let down
Thanks!
Ngl, my bro is the Jay-Z of LLM education...out here dropping bangers.
I just wish to be this talented- amazing video
I just want to see a 7B mamba model trained on the same data as a 7B transformer model and get to try them both and test them on certain abilities.
many of us want it too
Such a great video, and FUNNY! Glad you're making these
00:27 🔍 Self-attention mechanism in Transformers enables advanced text completion but struggles with basic arithmetic; companies integrate calculators to mitigate issues.
02:29 🔄 Mamba, a new model, addresses Transformer inefficiencies by scaling linearly, not exponentially, and doesn't rely on the attention mechanism.
05:32 🚀 Mamba potentially offers 1000x cheaper scaling than GPT-4, with quadratic scaling improvement and faster calculations, revolutionizing AI chatbots.
07:10 🧬 Mamba's long sequence handling benefits DNA modeling, audio synthesis, and analyzing high-resolution images or long-form videos.
10:29 💻 Mamba Bite model learns directly from raw bytes, eliminating tokenization biases, enhancing long sequence comprehension, and potentially enabling true multimodal models.
12:48 ⚠ Potential downside of Mamba: "Lost in the Middle" issue may lead to permanent loss of information in very long contexts; further research needed to address this.
I have no idea how LLMs work and I still understood some of it and the implications. Which is to say, this is a great video my dude.
Hardest AI channel 🪨 🤘
subbed...
what if there is a 70B mamba ? it can surpass existing ones? I don't saw in any place comparations where they compare mamba 70b with any other big model. perhaps it would be a decisive analysis to see how it performes
I assume that bigger models are still in the works. Most attention is on transformers-based models, so the money and resources to train a 70b model for Mamba are taking longer to gather. I'm definitely looking forward to seeing what becomes of it though.
that was a great part at 7:21
If you were one of the professors at my school I would never miss a class lol. You are great at breaking down concepts and making it funny. Keep up the good work!
Woooooowwww, 8ish minutes in was a mic drop I didn't expect. First time here, not the last.
Your videos are just the best. Humor and knowledge in its best combination
07:24 : This should be clipped and meme-d until all of mankind's Mamba stood erect and proud
Just wait until they discover Mamba No.5. There will be no going back...
Thanks for the papers in the description, I just put them a urls=[] and hydrated my s3 and vdb with them 5:42 😎
Dangit, now I have Lou Bega's Mambo No. 5 stuck in my head!
the JJK edit was INSANELY funny!
5:02 i am all in for non-sound memes. at least it doesnt make me a weirdo when watching without earphones in an open space.
GOD LEVEL EDIT, WHO IS THIS RUclipsR?
This video is where I will start with my thesis.
7:22 Fire 🔥🔥🔥🔥🔥
The LLM anime edit earned my subscription
Wtf did I just watch? This is like the best ever.
I cannot explain how much I enjoyed that edit
5:39 It's only 1000 times cheaper because the price was per 1000 tokens. If it was per 4000 tokens, it would get 4000 times cheaper, and so on. 😊
Yoo that LLM'S Anime edit hit hardddd😂😂
11:45 why can't we force it to process numbers as sequences of digits in certain contexts?
Plz tell me how do you select gifs for videos it's so precised that I doubt it's made by chatgpt no human
7:25mins in woah!!! I want all of my tech news delivered in this format!! excitingly eccentric, suspense filled, noir comic animations with deep, rich & sexy actor voice overs.. so cool! @bycloudAI
13:15 that’s the most passive paper name I’ve ever seen
the jjk edit killed me, bro i love you
I alone am the subquadratic one.
finally understood why AI cannot tell how many letters N word banana contains )
damn, I just tested it with gpt4, and it said that there are 3 n in banana.
It's so funny 🤣
@@Hollowed2wiz what is interesting is that gpt4 can spell this word one letter at time if asked and then give right answer about letter N count. So it seems that despite tokinezation gpt4 knows something about spelling...
The edit made me understand than without :D
As an electrical engineer: REAL TRANSFORMERS HAVE WINDINGS.
Bro. You make these LLM videos so interesting and funny. How do you come up with these? Keep it up
Didn't expect to see some jojo and gojo references in a AI model video, awesome!
These comments make me feel stupid 😭
All I understood is that mamba is faster than traditional transformer thing (the party explanation was awesome, thank you), and that mamba abandons the tokenization and uses... something else instead... I kinda wish you would summarize the video at the end for silly creatures like me
But for now it's time to rewatch everything!! 🥳
Bro the JJK edit is superb
I wonder if OpenAI switches to Mamba architecture if they will drop the "GPT" branding since technically the T will not apply anymore. I wonder if "GPT" will be like how Boomers would call every game console a "nintendo" and just used by the mainstream to mean every LLM, no matter the underlying architecture.
Lol Nintendo, but doesn't GPT = General Pre-Trained, not "-Transformer"?
VMamba has updated the scaling chart at 9:35. Performance keeps increasing with increased model size, but not much
tldr by Perplexity:
The video titled "Mamba Might Just Make LLMs 1000x Cheaper..." discusses a new AI architecture known as Mamba, which aims to significantly reduce the cost and improve the efficiency of Large Language Models (LLMs). Mamba differentiates itself from traditional Transformer models by utilizing a State Space Sequence Model (S4) and a selective mechanism, which allows for linear scaling and faster inference times. This architecture shows promise not only in language tasks but also in vision tasks, indicating its potential as a versatile tool in AI development. Mamba's approach to learning directly from raw byte patterns rather than tokenized text addresses the challenges posed by tokenization, such as distortion in text representation. This method enables Mamba to generate more accurate and coherent text, especially for long or complex sequences. Despite its advantages, Mamba faces challenges like information loss in long contexts. However, its introduction represents a significant advancement in AI architectures, potentially challenging the dominance of Transformers.
This and the LPU that Groq uses are going to be insane together.
That thing was still created purely for a Transformers style model.
They will need to make a new arch
yeah, I understand you mamba, I too have a "lost in the middle" problem
Without a serious upscaled mamba it will be going no were expect for niche areas
I expected "Are your hidden states linear because you're a State Space Model? Because I can't seem to figure out your next move."
:D
I've been attending parties with the mamba method forever. I may have been doing that wrong.
Imagine if the middle eastern investors invested trillions in obsolete technology
Even if this doesn't replace transformers, this looks like a very promising way to replace tokenization/word vectors by having a layer read the bytes and output vector tokens
Great video, I already heard about Mamba, but didn’t get into it myself! The lobotomy Kaisen edit went really hard haha.
Any chance you will be making a video about Liquid Neural Networks?
Keep up the good work :)
I would not have commented if it wasn't for that "nah I'd win" Mamba edit
I do love Mamba for faster token/sec. But there's still a long road to make it able to extract key information from long text. For now it still feels like Bart or Gemma 2b for short prompt
Great content. Keep up the great work! I subbed
Sooner or later we would need a bytes-level model architecture for multi-modality. Hope the test result for this one be good🙏 .
Cool video btw 👌
bro the content on this channel is for such a niche audience only 5 people on RUclips will get all the memes and understand the science
This whole video is Gold.
the LLM anime edit was fucking gold
Lobotomy kaisen edit was PEAK
Nice ! I wonder if this would be compatible with Groq's chip.
I imagine Transformer guys are seeing MAMBA as a big 2 way transformer and yank it in the Transformer architecture forming multi-architecture transformer model.
Wonder what Google are doing with 1M - 10M+ context Gemini Pro 1.5?
the video was good and i liked your style, but then the JJK edit dropped
actual fire 🔥
i hope you don't mind me reposting the JJK edit section of your video
tell me if you want me to take it down
Dude... We are dangerously close to a bearish breakout.
Fantastic sub. Need more neutral analysis of cutting edge
bro that anime thing was just incredible
Cool reference to the router edit
that gojo edit has to be the hardest thing ive seen in years wtf were you on when that came to your mind
How do you create such videos? Like what app? And what for b-rolls?
7:22 NOOOOOOO NOT AGAIN
mamba is snake in luganda
Can you build a Mamba based LLM or a multimodal language model using bitnet b1.58?