ّDear Pascal, I was wondering if you have any presentations to describe the article titled Neural Machine Translation By Jointly Learning To Align And Translate.
Wow, I was gonna ask, why didn't he explain or give an example for query, key, value in the case of a simple language translation or modelling example. Machine learning community is not very good at conveying their ideas, when you can't put stuff in rigorous mathematics at least use a lot of pictures and many examples at every possible step.
One of the best explanations for attention & transformers in RUclips. Most of the other videos are junk with authors pretending to understand the concepts and just adding to the RUclips clutter.
@@drdr3496 nice. Can you explain me whole query , key and value process for some example like " the cat sat on the mat". What is query, key and values for this sentence
In recent work, titled "Generalized Attention Mechanism and Relative Position for Transformer" , on transformer it is shown that different matrices for query and key are not required for attention mechanism in Transformer thus reducing number of parameters to be trained for Transformer of GPT, other language models and Transformers for images/videos.
wow! love the way he answers that great question at around 50:52 : “why we dont we implement the mask with hadamard product outside of the softmax?”. brilliant prof.
Thank you for the very comprehensive and understandable course. Being in 2024 I can say that I can understand even better and easier this course using GPT-4.
Does anyone know? Which NMT video has the previous intro to Attention the professor cites in this video? I couldn’t find his video on neural machine translation.
41:14 Question, on the output side, why isn't there an additional feed-forward layer between the masked self attention in the output and the attention to the input? And maybe more broadly what are those feed forward units doing?
Could you explain what the model class looks like then? What is the capacity? What is the "unconstrained" version with higher capacity? I was full statistical learning theory style discussion in all pedalogical discussions. I don't understand how people think they understand this. If your life depended on it, would you feel confident in recommending one of these setups? What questions would you have to ask about the data, the model architecture, the observation process? You need worst case bounds, model complexity etc. I see none of that here.
@@syphiliticpangloss Well, in deep learning the theory is far behind engineering. When people say they understand this lecture, they don't mean worst case bounds (which I strongly doubt anyone in the world knows how to calculate for this, without adding so many relaxation assumptions which make it basically irrelevant, like convexity etc.), they just mean that: 1. Engineering wise they understand how to build and use it 2. They feel they grasp enough intuition to what is the purpose of each sub-block and why it was added. I don't think anyone truly "understands" much simpler models in DL than transformers, which perform in a far superior level to classical machine learning methods. For example, fully convolutional neural networks, trained with Adam optimizer, based on back-propagation, using BN.
@@1Kapachow1 So can someone explain what the transformer is doing then in a precise way? I would accept answers that reference probability distributions and predictive goals or computation description of components like NAND gates etc. Also accepted would be anything related to the eigenvalues, stability, curvature etc. There are lots of people trying to talk about this stuff. For example arxiv.org/abs/2004.09280 Or Vapnick. To be perfectly clear, I think today we tend to say there are only two things really: a) "data" i.e. observations usually dozens to millions from some process we take to be slowly changing at most and b) predicates/models/architecture/constraints ... "observations" usually less than dozens, usually manually constructure (from other experiments and observations sets perhaps). To each of these we usually have some sort of "narrative" about where each came from, a way of describing it in some way to humans. The second thing is what I'm getting at. "Architecture" is a model constraint. If it is just pulled from thin air without undestanding the problem, the meta-problem etc, it is quite likely that there are buried problems, secret reasons for architecture choices that are not being disclosed or realised. Getting better at describing these models/arch/predicates is how we progress.
19:00 it's basically an exclusionary perceptron layer, isn't it? (also could be called fuzzy LUT) I'm sure it was used before for the attention emulation
Softmax is computationally expensive, I wonder if this can be replaced somehow with another function to produce probabilities since Softmax is present in many places in all the blocks of the transformer network.
Question about positional encoding. If we sum the Word Embedding (WE) with the Positional Encoding (PE) how does the model tell the difference between WE = 0.5, PE = 0.2 and WE = 0.4 and PE = 0.3 ? (Different words that are at different positions can yield the same value) Why not keep the PE separate from WE?
Around @29:50 mark, he first mentions that the key vectors correspond to each output word, but the slide mentions input word. Could someone please clarify this?
the explanation of K,V and Q is great. unfortunately i lost him as soon as he started on multi-head. must be that the single head i possess is empty :-)
Thanks for the nice lecture. I am still confused regarding how transformers model can replace RNNs or LSTMs for general sequence learning. The size of a sequence might be very lengthy in some applications rather than just a sentence (which can be designed to be fixed in length) so how to deal with this especially if we need to keep the complete sequence with us as there is no recurrence? If the answer is to divide the sequence, then how to link different chunks over time without a recurrence or a carry over? (Loops over time)
transformers are able to "query the recurrencies", think of it as instead of repeating the operation as in RNNs you just query 'x' times a database of the possible values and its given inputs and check if it matches the requirements, and because it's not a recurrence, repetition, you can make multiple of these queries, each being a new operation, at the same time!! each operation can be resolved without interference creating new tokens, or pieces, which represent convergence points in the data universe you are travelling. it's a huge improvement.. confirmed by the models shown at the end of the lecture. hope this helps :)
As JAKKO mentioned the transforms use the attention mechanism in a very efficient manner. The size of the sequence can be sufficiently longer than a sentence and still the attention mechanism will be able to capture the dependencies between the words at different positions. And this creates an efficient contextual representation of the sequence better than the normal input embedding vector. And this is how the complete input sequence is captured by the model without the recurrence. This is really a beautiful approach. (personal opinion)
45:50 - For the multiple linear transformations, are we applying the same linear transform to each set of Q/K/V in a "head" ? Or does each Q/K/V get its own unique linear transform applied?
Unique linear transform each time I guess.( In the original paper, under section 3.2.2 they mention that " h times with different learned linear projections to dk, dk and dv respectively") If we take repeat scaled dot product attention 3 times , then we will have total of 9 linear projections.
At 1:18:22, the professor refers to BERT and a "Decoder transformer that predicts a missing word". To me, BERT is a masked Encoder (not decoder). After all, BERT stands for bidirectional *encoder* representation from transformers. It's minor (and doesn't subtract from this great presentation), but can anyone comment?
Very intersting lecture. Something that is not totally clear on minute 46: these multihead presented intuitievely as explicit 3 various filterts as in cnn to produce 3 corresponding feature maps,but on previous part of lecture its being said that multi heads are stacked one after another to produce at first info from (word i,word j) and second pairs of that stuff i.e one is the input to another one. So how to understand it in the rightr way? Seems like on minute 46 the inputs to each of the linear are the same but on lecture part it looks like one is going after another and intuitevly the pair of pais and so one changes the ouput size.
I can comprehend the additive part as incrementing the knowledge from the self-attention operation to the query itself. You can think of it as the query (let's call it x) is a proper question and the sublayer (multihead attention) operation output is the contribution to your inquiry from the oracle (the sub-layer before the feed-forward step), which can then be used to refine your question (the query). If you look at the paper, it's really LayerNorm(x + Sublayer(x)) what the authors write for the Encoder and Decoder stacks.
1:02:00 For the positional embedding, I am confused... It seems like the formula produces a scalar output (ie you put in a position in your sequence, you get out some sin/cos value). How does it become a vector?
You could take the value for each dimension in your embedding and then you have a vector of values. But then it seems you'd be encoding the same information in every input, which doesn't seem helpful?
@@ryandruckman999 Actually while doing that the values fed into the cos and sin function also takes the scalar position of the word. So each positional embedding vector will have different values which in turn captures the position / order of the sequence. refer this article : kazemnejad.com/blog/transformer_architecture_positional_encoding/
Why do you think it is amazing? Can you explain exactly how transformers change the capacity, vc dimension, radamacher complexity, how they change the "hypothesis space" and is there an equivalent "regularization". Otherwise this is just explaining some construction with no understanding. I think they should always draw the "block" structure across the temporal dimension. This would make it clearer. So "Dense" connections vs attention layers etc. I don't get the sense anyone is actually trying to actually think about what this operator class looks like. Kind of disheartening.
@@karthikvenkatesh3538 Dunno, that is kind my point. Always start with an SDE description in terms of functions and then describe the approximation class. I think it would help everyone quickly absorb things. It is easier to talk about the spectrum of the Hessian and whatever else one might try to think about abstractly when building approximation class structure manually. I think things like this might get into it arxiv.org/pdf/2001.08317.pdf
The slides are posted here: cs.uwaterloo.ca/~ppoupart/teaching/cs480-spring19/schedule.html
Hello Professor. Your explanations are amazing. Kindly pin this comment or add in description so that it is visible to everyone.
ّDear Pascal, I was wondering if you have any presentations to describe the article titled Neural Machine Translation By Jointly Learning To Align And Translate.
Finally, Someone can explain simply what queries, keys , values are in the transformer model . Thank you Sir !!!
Yeah, I don't understand why nobody else goes over this seemingly pretty important detail
Wow, I was gonna ask, why didn't he explain or give an example for query, key, value in the case of a simple language translation or modelling example. Machine learning community is not very good at conveying their ideas, when you can't put stuff in rigorous mathematics at least use a lot of pictures and many examples at every possible step.
I can also recommend this explanation: ruclips.net/video/mMa2PmYJlCo/видео.html
It has helped me a lot
@@andrii5054 The video really simplify the concept. Thanks for sharing !
Where was that?
00:00 Attention
31:32 Transformer
47:15 Masked Multi-head Attention
1:01:45 Layer normalization, Positional embedding
Thank you for this!
thanks
One of the best explanations for attention & transformers in RUclips. Most of the other videos are junk with authors pretending to understand the concepts and just adding to the RUclips clutter.
Thank you very much Dr. Poupart. This is the best explanation of transformers I have come across on the internet
This is the best lecture on attention mechanism I can find! Thank you Dr. Poupart! finally all the details made sense to me.
This is the single best video on "Attention is all you need", attention, transformers, etc. on the Internet. It's simple as that. Thanks Dr Poupart.
Why does a dot product correlate to attention?
@@bleacherz7503 a dot product between two vectors shows how similar they are
@@drdr3496 nice. Can you explain me whole query , key and value process for some example like " the cat sat on the mat". What is query, key and values for this sentence
ty
Best video on attention that I have come across
Thank you very much, this was by far the best explanation of Transformers that I found online!
I was about to type the same.
By far the best I have seen to explain multiheaded attention
Thank you to the person asking the question at 28:49! The softmax gave it away, but I wasn't sure.
Very enjoyed your lecture, Professor Poupart! Very informative and simplified many complicated concepts. Thank you very much!
This is the best Transformer / Attention Explaination ever.. Thank you
This video just cured my depression.
dont worry, im sure the next visit to a public internet forum will once again obliterate hope in humanity
haha same here
Cured my ADHD
This video gave me depression…
@@Mrduralexx yeah its a different level of anxiety that it instills
In recent work, titled "Generalized Attention Mechanism and Relative Position for Transformer" , on transformer it is shown that different matrices for query and key are not required for attention mechanism in Transformer thus reducing number of parameters to be trained for Transformer of GPT, other language models and Transformers for images/videos.
Most awesome video on transformers one could find on youtube
one of the best and detailed lecture on attention on youtube I must say.
You are a blessing, finally understood a very important concept.
wow! love the way he answers that great question at around 50:52 : “why we dont we implement the mask with hadamard product outside of the softmax?”. brilliant prof.
Thanks a lot Professor Poupart one of the best explanation for maths behind transformers!
A great lecture! Highly recommended!
One of the best videos!
Thank you for the clear explanation and well-illustrated examples!
Thank you for the very comprehensive and understandable course. Being in 2024 I can say that I can understand even better and easier this course using GPT-4.
thank you for being so thorough!
I am not able to find a better video than this one explaining attention and transformer on the internet
I totally agree, He is genius!
The best explanation of attention mechanism I found on RUclips is the making pizza analogue by Alfredo Canziani.
I eventually understood the Q,K,V in attention. Many thanks.
His toturial video is my favorite timepass, specially at night,Hail to prof. Poupart
A very interesting explanation (wonderful)
Bhaiji love from India. Is lecture ko araam se dekhunga
I would call this the best explanation of attention/transformers on youtube i have found so far.
This is the best video Internet about Transformers network
Best explanation of transformer I have ever seen, thank you Dr Poupart
Nice lecture! Thanks!
Cool lecture!
Truly fantastic wow
finallu understood what is going on . most of the videos are so simple and skipped math . i liked it
This was a great lecture, thank you.
thank you, the video is really useful. 👍🏻👍🏻
The simplest explanation of attention, thanks a lot for sharing, great lectures🤗!
I asked chatGPT for the best video on Attention and it brought me here
Thank you so much, the brainstorming
thank you my man, u saved me
Thank you for this :)
Very good explanation. Thanks.
Great video!
very interesting and useful. Thanks Sir
Does anyone know? Which NMT video has the previous intro to Attention the professor cites in this video? I couldn’t find his video on neural machine translation.
Beautiful explanations
Thank you Professor :)
41:14 Question, on the output side, why isn't there an additional feed-forward layer between the masked self attention in the output and the attention to the input? And maybe more broadly what are those feed forward units doing?
Big fan, big fan Sir!!
Finally understood this!
True
Thank you very much. It is very detailed and captures the intuition.
Could you explain what the model class looks like then? What is the capacity? What is the "unconstrained" version with higher capacity? I was full statistical learning theory style discussion in all pedalogical discussions. I don't understand how people think they understand this.
If your life depended on it, would you feel confident in recommending one of these setups? What questions would you have to ask about the data, the model architecture, the observation process? You need worst case bounds, model complexity etc. I see none of that here.
@@syphiliticpangloss Well, in deep learning the theory is far behind engineering.
When people say they understand this lecture, they don't mean worst case bounds (which I strongly doubt anyone in the world knows how to calculate for this, without adding so many relaxation assumptions which make it basically irrelevant, like convexity etc.),
they just mean that:
1. Engineering wise they understand how to build and use it
2. They feel they grasp enough intuition to what is the purpose of each sub-block and why it was added.
I don't think anyone truly "understands" much simpler models in DL than transformers, which perform in a far superior level to classical machine learning methods.
For example, fully convolutional neural networks, trained with Adam optimizer, based on back-propagation, using BN.
@@1Kapachow1 So can someone explain what the transformer is doing then in a precise way? I would accept answers that reference probability distributions and predictive goals or computation description of components like NAND gates etc.
Also accepted would be anything related to the eigenvalues, stability, curvature etc.
There are lots of people trying to talk about this stuff. For example arxiv.org/abs/2004.09280
Or Vapnick.
To be perfectly clear, I think today we tend to say there are only two things really: a) "data" i.e. observations usually dozens to millions from some process we take to be slowly changing at most and b) predicates/models/architecture/constraints ... "observations" usually less than dozens, usually manually constructure (from other experiments and observations sets perhaps). To each of these we usually have some sort of "narrative" about where each came from, a way of describing it in some way to humans.
The second thing is what I'm getting at. "Architecture" is a model constraint. If it is just pulled from thin air without undestanding the problem, the meta-problem etc, it is quite likely that there are buried problems, secret reasons for architecture choices that are not being disclosed or realised.
Getting better at describing these models/arch/predicates is how we progress.
Great lecture! Thank you Professor!
Little did anyone know just how groundbreaking this foundation would be for Chat GPT / GPT 4.
Great explanation sir... Thank You! Please post more such lectures and I would be the first to look at it...
Excellent session....Tq professor
I'm a novice and have been praying to get someone who discusses these papers. You're an answered prayer! Great lecturer.
Great lecture.
Thank you so much for great explanation, professor.
Beautiful.
Beautifully explained.. things seem clear to me now .. Thanks a lot sir!
Very clear explanation
Thank you very much Sir!
19:00 it's basically an exclusionary perceptron layer, isn't it? (also could be called fuzzy LUT) I'm sure it was used before for the attention emulation
Softmax is computationally expensive, I wonder if this can be replaced somehow with another function to produce probabilities since Softmax is present in many places in all the blocks of the transformer network.
Woohoo... Thank you so much
Clearly! Thanks
great explanation
Best explanation
Thank you soo much!!!
Thank you!
Best video to watch on the first day of 2023
Thank You.
very good video!😀
Huge thanks for putting this online.
Great point at 50:50 👍
Question about positional encoding.
If we sum the Word Embedding (WE) with the Positional Encoding (PE) how does the model
tell the difference between WE = 0.5, PE = 0.2 and WE = 0.4 and PE = 0.3 ?
(Different words that are at different positions can yield the same value)
Why not keep the PE separate from WE?
I heard queries, keys & values were primative concepts and counter-intuitive, but I didn't know it was THIS primative.
This was a great lecture - really explained this to me thanks
Around @29:50 mark, he first mentions that the key vectors correspond to each output word, but the slide mentions input word. Could someone please clarify this?
This year should be the "tranformer year" as there a breakout in domain of CV.
the explanation of K,V and Q is great. unfortunately i lost him as soon as he started on multi-head. must be that the single head i possess is empty :-)
46:45 - Also, the output shape is the same as the input shape right? ie, the size of the input sequence?
Thanks for the nice lecture. I am still confused regarding how transformers model can replace RNNs or LSTMs for general sequence learning. The size of a sequence might be very lengthy in some applications rather than just a sentence (which can be designed to be fixed in length) so how to deal with this especially if we need to keep the complete sequence with us as there is no recurrence? If the answer is to divide the sequence, then how to link different chunks over time without a recurrence or a carry over? (Loops over time)
transformers are able to "query the recurrencies", think of it as instead of repeating the operation as in RNNs you just query 'x' times a database of the possible values and its given inputs and check if it matches the requirements, and because it's not a recurrence, repetition, you can make multiple of these queries, each being a new operation, at the same time!! each operation can be resolved without interference creating new tokens, or pieces, which represent convergence points in the data universe you are travelling.
it's a huge improvement.. confirmed by the models shown at the end of the lecture. hope this helps :)
As JAKKO mentioned the transforms use the attention mechanism in a very efficient manner. The size of the sequence can be sufficiently longer than a sentence and still the attention mechanism will be able to capture the dependencies between the words at different positions. And this creates an efficient contextual representation of the sequence better than the normal input embedding vector. And this is how the complete input sequence is captured by the model without the recurrence.
This is really a beautiful approach. (personal opinion)
45:50 - For the multiple linear transformations, are we applying the same linear transform to each set of Q/K/V in a "head" ? Or does each Q/K/V get its own unique linear transform applied?
Unique linear transform each time I guess.( In the original paper, under section 3.2.2 they mention that " h times with different learned linear projections to dk, dk and dv respectively")
If we take repeat scaled dot product attention 3 times , then we will have total of 9 linear projections.
They each get their own learned parameters.
and thats how gpt was born my fellas
Good explanation
thanks professor for easy explain ... can you share powerpoint with us ..
Very helpful
At 1:18:22, the professor refers to BERT and a "Decoder transformer that predicts a missing word".
To me, BERT is a masked Encoder (not decoder).
After all, BERT stands for bidirectional *encoder* representation from transformers.
It's minor (and doesn't subtract from this great presentation), but can anyone comment?
Great lecture. And I believe you are correct, it seems there is a typo here. I was questioning the same!
Thank you very much for sharing that great lecture!
Shouldn't it be the attention vector instead of the value? at 27:44
Very intersting lecture. Something that is not totally clear on minute 46: these multihead presented intuitievely as explicit 3 various filterts as in cnn to produce 3 corresponding feature maps,but on previous part of lecture its being said that multi heads are stacked one after another to produce at first info from (word i,word j) and second pairs of that stuff i.e one is the input to another one. So how to understand it in the rightr way? Seems like on minute 46 the inputs to each of the linear are the same but on lecture part it looks like one is going after another and intuitevly the pair of pais and so one changes the ouput size.
does this normalization layer matter in the inference stage?
57:30 - I understand the normalization, but what's the intuition for adding? Does it just strengthen the signal from the input sequence?
I can comprehend the additive part as incrementing the knowledge from the self-attention operation to the query itself. You can think of it as the query (let's call it x) is a proper question and the sublayer (multihead attention) operation output is the contribution to your inquiry from the oracle (the sub-layer before the feed-forward step), which can then be used to refine your question (the query). If you look at the paper, it's really LayerNorm(x + Sublayer(x)) what the authors write for the Encoder and Decoder stacks.
Saviour!!!
1:02:00 For the positional embedding, I am confused... It seems like the formula produces a scalar output (ie you put in a position in your sequence, you get out some sin/cos value). How does it become a vector?
You could take the value for each dimension in your embedding and then you have a vector of values. But then it seems you'd be encoding the same information in every input, which doesn't seem helpful?
@@ryandruckman999 Actually while doing that the values fed into the cos and sin function also takes the scalar position of the word. So each positional embedding vector will have different values which in turn captures the position / order of the sequence.
refer this article : kazemnejad.com/blog/transformer_architecture_positional_encoding/
Thank you sir
Amazing lecture! I am learning a lot! I have some questions though:
Hey Ryan, you can join one of the online events in this meetup group ;)
www.meetup.com/Study-Of-Advanced-Natural-Language-Processing-Topics/
Why do you think it is amazing? Can you explain exactly how transformers change the capacity, vc dimension, radamacher complexity, how they change the "hypothesis space" and is there an equivalent "regularization". Otherwise this is just explaining some construction with no understanding.
I think they should always draw the "block" structure across the temporal dimension. This would make it clearer. So "Dense" connections vs attention layers etc. I don't get the sense anyone is actually trying to actually think about what this operator class looks like. Kind of disheartening.
@@syphiliticpangloss can you point me towards videos or readings with the transformer block across the temporal dimension?
@@karthikvenkatesh3538 Dunno, that is kind my point. Always start with an SDE description in terms of functions and then describe the approximation class. I think it would help everyone quickly absorb things. It is easier to talk about the spectrum of the Hessian and whatever else one might try to think about abstractly when building approximation class structure manually.
I think things like this might get into it arxiv.org/pdf/2001.08317.pdf