Thanks for the explanation. At 9:19 : Shouldn't the order of multiplication be the opposite here? E.g. x1(vector) * Wq(matrix) = q1(vector). Otherwise I don't understand how we get the 1x3 dimensionality at the end
Awesome video, thank you! I love the idea of you revisiting older topics -- either as a 201 or as a re-introduction. "Attention combines the representation of input vector's value vectors, weighted by the importance score (computed by the query and key vectors)."
Wow, this explanation on the difference between RNNs and Transformers at the end is what I was missing! I've always heard that Transformers are great because of parallelization but never really saw why until today, thank you! Great video!
You know how to explain things. This one is not easy: I can see the amount of work that went into this video, and it was a lot. I hope that your career takes you where you deserve.
Thank you very much for the very clear explanations and detailed analysis of the transformer architecture. Your truly the 3blue1brown of machine learning!
Time is quadratic, but memory is linear -- see the FlashAttention paper. But the number of parameters is constant -- that's the magic ! Thanks for the excellent videos ! 👍
Thanks so much for this video. I’ve gone through a number of videos on transformers and this is much easier to grasp and understand for a non-data scientist like myself.
Tomorrow i have thesis evaluation and i was thinking about watching that video again, but youtube algorithm suggested me without searching anything, Thank u youtube algo.. 😅❤🔥
This is a very well-made explanation. I hadn't known that the feedforward layers only received one token at a time. Thanks for clearing that up for me! 😁
Today, I had the problem I need to understand how Transformers work.. I searched on youtube and found your video 20 minutes after release. What a perfect timing
In 11:14, the weighted sum is the sum of 3 vectors of 3 elements each, but the results is a vector of 4 elements. Which, conveniently, is the same size as the input vector. Could there be a missing step there?
As far as I am aware, word embedding has changed from legacy static embedding like Word2Vec/GLOVE (like the famous queen=woman+king-man metaphor) to BPE & unigram, this change gave me quite a headache, as most of paper do not mention any detail of their "word embedding". Perhaps Letitia you can make a video to clarify this a bit for us.
that's great, I'm a little stuck on the special mask token? ... I'll keep digging, good info, the video is good explanation, it allows for more experimentation instead of relying on open source models that can have components look like a black box to noobs like me :)
❤ Letitia, thank you for great visualization and intuition. For inspiration: In the original paper, the decoder utilizes the output of the encoder by running a cross-attention process. Why does GPT not use an encoder? As you've mentioned, the encoder is typically used for classification, while the decoder is for text generation. They are never used in combination. Why is this the case? Missing Intuition: Why does the cross-attention layer inside the decoder take the values from the ENCODER’s output to create the enhanced embeddings (as a weighted mix)? Intuitively, I would use the values from the DECODER.
Thanks for your prompt reply. Hence, understanding the concept and intuition behind feeding the encoder output into the decoder is essential. I found only this one video on encoder-decoder cross-attention: ruclips.net/video/Dqjq4Gxdhng/видео.htmlsi=gtLzNxAU0pUGyLvk In it, Lennart emphasizes the observation that, based on the original equations, we have the enhanced embeddings calculated as a weighted sum of ENCODER values. Inside of a DECODER, I would rather expect to have the DECODER values pass through. Letitia, I am sure, you will resolve this mystery. 🍀
Good explanation. Most videos on attention focus on how it's implemented, not on the design choices behind it. To my understanding, the goal was to mitigate the computational inefficiencies of RNNs and the spatial limitations of CNNs in order to achieve a universal representation of a sequence. I wanted to clarify one thing: you depicted multiple FFNNs similarly to how RNNs are usually rolled out. Is it just the same one FFNN that takes a single attention-encoded vector as input and predicts the next token from this ONE vector? By the way, what brand is that sweater? Loro Piana? :)
What a great video. It still could expand more and really sum up every sub-part and connect it to a certain clear visualization or clear step of what happens with the information at each time step and how its "transformation" progresses over time. So i think you could redo this video and really make it monkey proof for folks like me. But beware, if you look for example at the StatQuest version, its to slow and too repetative and also does not really capture, what really goes on inside the Transformer, once all steps are stacked together. Great work!
Thanks for your video. I have a question on inference process. For example when I have a input prompt of 2 tokens = {t1, t2}. we will get the output {o1, o2, o3}. we will take only o3 and make new input sequence {t1, t2, o3}. Then we will get another output {o'1, o'2, o'3, o'4}. Here my questions are. When we use causal masking for attention, o1= o'1 and o2=o'2 and so on? Another question, even though the mask guarantee the causal attension. but still the matrix calcuation is performed. Then it means the computation is used any way. How can we reduce the computation resource for this case.
hi! can i ask question of how did you get the dimension (d)? because all i know is dimension can be found in square matrices, and the dot product of the attention formula says that Q•K^T. if we're using 1x3 matrices, we'll get 1x1 matrices or 1 dimension, how do you get 3 ? unless its 3x1 matrix beforehand, so we'll get 3x3 or 3 dimensional matrix. thankyouu !
Hi, if you mean the mistake at 10:00, then the problem is that I have written matrix times vector when I should have written vector times matrix! (or I could have used column vectors instead of row vectors). Is this what you mean?
Everything makes sense except multiple attention heads. Each layer has only one set of Q, K, V, O matrices. But 8 attention heads per layer? I want to understand that.
Think about it this way: In one layer, instead of having one head telling you how to pay attention at things, you have 8. In other words, instead of having one person shout at you the things they want you to pay attention to, you have 8 people simultaneously shouting at you. This is beneficial because it has an ensembling effect (the effect of a voting parliament. Think of Random Forests that are an ensemble of Decision Trees). I do not know if this helps, but I thought giving it another shot at explaining this.
![The moment we stopped understanding AI [AlexNet]](http://i.ytimg.com/vi/UZDiGooFs54/mqdefault.jpg)
![Felix "Unfair" | [Stray Kids : SKZ-PLAYER]](http://i.ytimg.com/vi/Oswujxm2Ag0/mqdefault.jpg)
Thanks for the explanation. At 9:19 : Shouldn't the order of multiplication be the opposite here? E.g. x1(vector) * Wq(matrix) = q1(vector). Otherwise I don't understand how we get the 1x3 dimensionality at the end
Oh, shoot, messed up the order in the animations there. You are right. Sorry, pinning your comment.
No problem thanks for clarifying that, and thanks again for the great video@@AICoffeeBreak
The vectors should be column vectors.
Awesome video, thank you! I love the idea of you revisiting older topics -- either as a 201 or as a re-introduction. "Attention combines the representation of input vector's value vectors, weighted by the importance score (computed by the query and key vectors)."
Thanks for your appreciation!
Epic as always 🤌
Thanks, Tim!
One of the best videos on transformers that I have ever watched. Views 📈
Do you have examples of others you liked?
Wow, this explanation on the difference between RNNs and Transformers at the end is what I was missing!
I've always heard that Transformers are great because of parallelization but never really saw why until today, thank you! Great video!
Oh, this makes me happy !
BEST of BEST Explanation. 1) Visually, 2) intuitively, 3) by numerical examples. And your English is better than native for Foreigners to listen.
Wow, you've come a long way since your first transformer explained video!
Best didatic explanation about Transformers so far. Thank you for sharing it.
Wow, thanks! Glad it's helpful.
You know how to explain things. This one is not easy: I can see the amount of work that went into this video, and it was a lot. I hope that your career takes you where you deserve.
Thanks for watching and thanks for the kind words. All the best to you as well!
Thank you very much for the very clear explanations and detailed analysis of the transformer architecture. Your truly the 3blue1brown of machine learning!
your videos has helped visual learner like me so much, thank you
Happy to hear that!
Absolute banger of a video. Wish I had seen this when I was learning about transformers in uni last year :-)
Haha, glad I could help. Even if a bit late.
Time is quadratic, but memory is linear -- see the FlashAttention paper.
But the number of parameters is constant -- that's the magic !
Thanks for the excellent videos ! 👍
Super informative and helpful! Thanks a lot!
Oh wow, thanks!
Understood about 10%, but I like these vidoes and feel intuitively the usefulness.
Thanks so much for this video. I’ve gone through a number of videos on transformers and this is much easier to grasp and understand for a non-data scientist like myself.
You're very welcome!
Letitia, you're awesome and I look forward to learning more from you.
What a wonderful explanation😍 Just discovered your channel and absolutely loving the explanations as well as visuals😘
Thank you! welcome!
Thank you. Nice explanation 😊
Thank You for your visit!
Really appreciate this video.
So glad!
I think I had at least 10 aha moments watching this, and I've watched many videos on these topics. Incredible job, thank you!
Wow, thank You for this wonderful comment!
What a wonderful video! Thank you so much for sharing it!
Thank you too for this wonderful comment!
Tomorrow i have thesis evaluation and i was thinking about watching that video again, but youtube algorithm suggested me without searching anything, Thank u youtube algo..
😅❤🔥
It read your mind.
Awesome! Epic Visuals!
Thanks, Connor!
Great Video!! Nice improvement over the original
Glad you think so!
Thanks for the explanation 😊
Thanks for watching!
This is a very well-made explanation. I hadn't known that the feedforward layers only received one token at a time. Thanks for clearing that up for me! 😁
This was really great, never really understood query, key and values before.
Thank you!
Really well done and easy to follow, thank you
Glad you enjoy it!
Great explanation -- loving your videos.The time codes for specific topics is really useful.
Thank you!
Today, I had the problem I need to understand how Transformers work.. I searched on youtube and found your video 20 minutes after release. What a perfect timing
What a timing!
Amazing explanation, Thank you! Just discovered your channel and I really like how the difficult topics are demystified.
Thanks a lot!
Great video and a good explanation. Thanks for your hard work on this amazing video!!
Glad you liked it!
Thank you for the video! Maybe an explanation on the Mamba Architecture next?
The Mamba and SSM beans are roasting as we speak.
Thanks, you helped so much explain Transformers to my PhD advisors
This is really funny. In what field are you doing your PhD? 😅
In 11:14, the weighted sum is the sum of 3 vectors of 3 elements each, but the results is a vector of 4 elements. Which, conveniently, is the same size as the input vector. Could there be a missing step there?
Yes, there is a missing back transformation to 4 dimensions I skipped. :) Well spotted!
As far as I am aware, word embedding has changed from legacy static embedding like Word2Vec/GLOVE (like the famous queen=woman+king-man metaphor) to BPE & unigram, this change gave me quite a headache, as most of paper do not mention any detail of their "word embedding". Perhaps Letitia you can make a video to clarify this a bit for us.
Great suggestion, thanks!
that's great, I'm a little stuck on the special mask token? ... I'll keep digging, good info, the video is good explanation, it allows for more experimentation instead of relying on open source models that can have components look like a black box to noobs like me :)
well explained . as you promised
Just discovered your channel and this is great! Thank you! :D
Thank you! Hope to see you again soon in the comments.
Had to go back and rewatch a section after I realized I'd been spacing out staring at the coffee bean's reactions.
here we go!
TY for content
Excellent explanation. Thank you!
Thank you! Finally, I start to get it...
How about a video explaining SSMs?
✍️
Psst: This will be the video coming up in a few days. it's in editing right now.
Yaay! @@AICoffeeBreak
great video
Thanks for the visit and for leaving the comment!
Thank you, Letitia!
Our pleasure!
...ok. After binging some of your vids, I now need to go make coffee. 😆
Please do!
❤ Letitia, thank you for great visualization and intuition. For inspiration: In the original paper, the decoder utilizes the output of the encoder by running a cross-attention process. Why does GPT not use an encoder? As you've mentioned, the encoder is typically used for classification, while the decoder is for text generation. They are never used in combination. Why is this the case?
Missing Intuition: Why does the cross-attention layer inside the decoder take the values from the ENCODER’s output to create the enhanced embeddings (as a weighted mix)? Intuitively, I would use the values from the DECODER.
Thanks for your thoughts! Encoders are sometimes used in combination with decoders, right? The most famous example is the T5 architecture.
Thanks for your prompt reply. Hence, understanding the concept and intuition behind feeding the encoder output into the decoder is essential. I found only this one video on encoder-decoder cross-attention:
ruclips.net/video/Dqjq4Gxdhng/видео.htmlsi=gtLzNxAU0pUGyLvk
In it, Lennart emphasizes the observation that, based on the original equations, we have the enhanced embeddings calculated as a weighted sum of ENCODER values. Inside of a DECODER, I would rather expect to have the DECODER values pass through.
Letitia, I am sure, you will resolve this mystery. 🍀
Awesome video :) thanks!
Thank you for watching and for your wonderful comment!
Oh my goddess in disguise, I thank you for saving me from depths of hell. Lots of love
Glad to help. 😆
Good explanation. Most videos on attention focus on how it's implemented, not on the design choices behind it. To my understanding, the goal was to mitigate the computational inefficiencies of RNNs and the spatial limitations of CNNs in order to achieve a universal representation of a sequence. I wanted to clarify one thing: you depicted multiple FFNNs similarly to how RNNs are usually rolled out. Is it just the same one FFNN that takes a single attention-encoded vector as input and predicts the next token from this ONE vector? By the way, what brand is that sweater? Loro Piana? :)
Phew 😳
You’re the best 👏👏👏
What a great video.
It still could expand more and really sum up every sub-part and connect it to a certain clear visualization or clear step of what happens with the information at each time step and how its "transformation" progresses over time.
So i think you could redo this video and really make it monkey proof for folks like me.
But beware, if you look for example at the StatQuest version, its to slow and too repetative and also does not really capture, what really goes on inside the Transformer, once all steps are stacked together.
Great work!
Thanks for your video. I have a question on inference process. For example when I have a input prompt of 2 tokens = {t1, t2}. we will get the output {o1, o2, o3}. we will take only o3 and make new input sequence {t1, t2, o3}. Then we will get another output {o'1, o'2, o'3, o'4}.
Here my questions are. When we use causal masking for attention, o1= o'1 and o2=o'2 and so on? Another question, even though the mask guarantee the causal attension. but still the matrix calcuation is performed. Then it means the computation is used any way. How can we reduce the computation resource for this case.
hi! can i ask question of how did you get the dimension (d)? because all i know is dimension can be found in square matrices, and the dot product of the attention formula says that Q•K^T. if we're using 1x3 matrices, we'll get 1x1 matrices or 1 dimension, how do you get 3 ? unless its 3x1 matrix beforehand, so we'll get 3x3 or 3 dimensional matrix.
thankyouu !
Hi, if you mean the mistake at 10:00, then the problem is that I have written matrix times vector when I should have written vector times matrix!
(or I could have used column vectors instead of row vectors). Is this what you mean?
Everything makes sense except multiple attention heads. Each layer has only one set of Q, K, V, O matrices. But 8 attention heads per layer? I want to understand that.
Think about it this way: In one layer, instead of having one head telling you how to pay attention at things, you have 8.
In other words, instead of having one person shout at you the things they want you to pay attention to, you have 8 people simultaneously shouting at you.
This is beneficial because it has an ensembling effect (the effect of a voting parliament. Think of Random Forests that are an ensemble of Decision Trees).
I do not know if this helps, but I thought giving it another shot at explaining this.
I want to train a transformer that eats a row of matrices instead of just a row of vectors.
Terrible
If you don't like her videos, why do you keep coming back to them just to comment that you didn't like it? Just watch something else.