I have recently posted another video which is even the detailed video on vision transformer. You can check that to understand the concepts in depth: ruclips.net/video/SIkYp6dscLw/видео.html
@@CodeWithAarohi i have a doubt arohi u made 3 videos on vit so also u did 1.11 hr video on vit is both are same or we differ from each other? can u clarify please thank you
@@Rakesh_Seerla They are different. 1.11 hr video is detailed video where concepts like linear projection, query, key and value are explained in detail.
@CodeWithAarohi nailed it. A question was asked about vit which I properly explained with diagram, thanks to your teaching. Keep making videos on ai topics. It really helps. Will explore your channel more for learning new topics. Thank you again ma'am.
Hello Ma’am Your AI and Data Science content is consistently impressive! Thanks for making complex concepts so accessible. Keep up the great work! 🚀 #ArtificialIntelligence #DataScience #ImpressiveContent 👏👍
The content is amazing! Very informative, short, and to the point, which is great for beginners. Thank you for these amazing videos 😍I have only one small feedback for your future videos. The audio quality is a little bit bad and noisy. You might consider checking your microphone.
Hi, it is nice content btw I have one doubt. If u divide 224*224 image into patch size = 16, that means there will be 16 grids as shown in 2:33 each patch having 14 pixels? Is my understanding correct?
You have a total of 14×14=196 patches. Each patch is 16×16 pixels in size. So, the number of grids (patches) is 196, and each patch has 16x16 pixels along each side, not 14x14.
@@CodeWithAarohi Oh okay. I got confused by looking onto image which had 16 cells. I hope that's a mistake right it should have been 14 cells right? And each cell has 16*16 pixels along its width and height
Matrix multiplication! Let's assume an image is split into 3x3 pixel and each pixel has 16x16 vector embedding which is flattened to 256x1 (means 256 rows and 1 column). Because we have 3x3 pixel size of image it means we have total 9 pixels. Hence if we combine the vector embedding of all the pixels (means if each pixel embedding is 256x1, then for 9 pixels it will become 256x9 i.e 256 rows and 9 columns. Now we have to pass this through linear layer. Linear layer let's say has 5 neurons. so shape for each neuron will be 256 x 1 (means 256 rows and 1 column) and for 5 neurons it will become 256x5 (menas 256 rows and 5 columns). Now we have to do matrix Multiplication of Input with Linear layer, but wait, we cannot multiply the matrix because shape of input is 256x9 and shape of linear is 256x5. In order to multiply the matrices, the columns of Matrix A must be equal to the number of rows of Matrix B. So we will transpose the input matrix of shape 256x9 to 9x256. Now, Let's take this as Matrix A of 9x256 and Matrix B of size 256x5. Because column of Matrix A is same as row of Matrix B, hence, dot product is possible which will result in new matrix of size 9x5. We can see that the original matrix of patch was of size 9x256 which is reduced to 9x5. So we will get the 3 matrices of size 9x5 each for Key, Query and Value. Now based on attention model we can see that we have to do the matrix multiplication of Query and Key and to do so we again have to do the transpose of Matrix because both Key and Query are of same shape (Query Matrix - 9x5 , Key Matrix - 9x5). So if we transpose Key Matrix it will become 5x9 and then the matrix multiplication will be possible between these two matrices (9x5 and 5x9). The dot product output of these matrices will be a matrix of size (9x9) and this output matrix is called as Attention Filter. Then after training we have the final updated values of this attention filter which we have to scale between 0 and 1 using softmax activation function. This scaled attention matrix (9x9) is then multiplied with Value matrix (9x5) which will give the filtered value of Matrix (9x5). Hence based on attention matrix we get the important feature of an image. This is the process of single attention head to extract feature. We use multi-head attention to extract various important features of an image. Each head focuses on different combinations of features.
The terms "Vanilla Vision Transformers" and "Vision Transformers" are often used interchangeably, and both refer to the same fundamental concept which is applying the Transformer architecture directly to image data for computer vision tasks.
the video was awesome . And can i know the transformer model of all the 6 encoders and 6 decoders , as I have confusion in the input architecture of decoders . Thank you mam
While ViT doesn't inherently provide feature importance scores like some other models, you can analyze the importance of different features in the classification process by examining the attention maps generated by the model. Attention maps in ViT represent the importance of each image patch in relation to the final prediction. Higher attention values indicate greater importance. By visualizing these attention maps, you can gain insights into which image regions contribute most to the classification decision.
Because in original paper of ViT, they have used patch size 16*16. You can work with some other patch size also. Eg: Patch Size 8×8 - which will give you 784 patches if your image size is 224*224. Another example: Patch Size 32×32 will give you 49 patches if image size is 224*224
I came to this video to learn how to do positional encoding for 2D images -- the precise math. When you come to that portion, you simply reference your intro video, re Transformers for linear text (and in which even the linear positional encoding isn't really explained).
@@CodeWithAarohi Thank you. Your method of teaching is amazing. But i am never comfortable with torch. Tensorflow is so natural for deep learning. I look forward to this .
Your explanation is one for the best I’ve heard about ViT, thank you very much
Glad it was helpful!
i agree
I am a new learner of vision transformer and your explaination is so simple to understand and is also so informative! Thank you!
Glad it was helpful!
I have recently posted another video which is even the detailed video on vision transformer. You can check that to understand the concepts in depth: ruclips.net/video/SIkYp6dscLw/видео.html
@@CodeWithAarohi i have a doubt arohi u made 3 videos on vit so also u did 1.11 hr video on vit is both are same or we differ from each other? can u clarify please thank you
@@Rakesh_Seerla They are different. 1.11 hr video is detailed video where concepts like linear projection, query, key and value are explained in detail.
Aarohi, I am watching you for 3 years now, and each time I understand the subject. You're literally the best
Thank you so much for your incredibly kind words! It means a lot to me😊
Thanks again for this very well explained tuto.
Glad it was helpful!
short and crisp. studying one day before exam. thank you ma'am
You are welcome. Hope your exam went well :)
@CodeWithAarohi nailed it. A question was asked about vit which I properly explained with diagram, thanks to your teaching. Keep making videos on ai topics. It really helps. Will explore your channel more for learning new topics. Thank you again ma'am.
V helpful for beginners❤
Glad this helped you!
What's about the extra class? and i think that only the extra class is used for the classification. Please could you explain this point?
Your tutorials are always the best, thank you very much. I hope you would create tutorials on Segformer soon.
Thank you, I will
Explanation is very clear .. excellent!
Glad it was helpful!
Hello Ma’am
Your AI and Data Science content is consistently impressive! Thanks for making complex concepts so accessible. Keep up the great work! 🚀 #ArtificialIntelligence #DataScience #ImpressiveContent 👏👍
My pleasure 😊
Great....crystal clear the concepts greatly explained 😊
Glad it helped!
The content is amazing! Very informative, short, and to the point, which is great for beginners. Thank you for these amazing videos 😍I have only one small feedback for your future videos. The audio quality is a little bit bad and noisy. You might consider checking your microphone.
Thank you for the feedback. I will take care of noise.
beautifully explained!
Glad it was helpful!
Thanks for this vedio.this tutorial is very clear and explaining and we had learning to how to split the pattern
You are welcome 😊
Great insights and easy to consume.
Glad it helped!
Very nice Presentation
Thanks a lot
you are a genius ma Shaa Allah, thanks a lot
You are most welcome
Hi, it is nice content btw I have one doubt.
If u divide 224*224 image into patch size = 16, that means there will be 16 grids as shown in 2:33 each patch having 14 pixels? Is my understanding correct?
You have a total of 14×14=196 patches. Each patch is 16×16 pixels in size. So, the number of grids (patches) is 196, and each patch has 16x16 pixels along each side, not 14x14.
@@CodeWithAarohi Oh okay. I got confused by looking onto image which had 16 cells. I hope that's a mistake right it should have been 14 cells right? And each cell has 16*16 pixels along its width and height
Thanks very much the videos are awesome and genuine.
Glad you like them!
very good explanation.
Very well explained. Thanks alot
Glad it was helpful!
very good explanation. Thank you
You are welcome!
Thank You, can you explain difference between CNN and ViT side by side.
How dimension is reduced for each 1D vector when each pixel of 1D vector is multiplied by weights? Can u clear it?
Matrix multiplication!
Let's assume an image is split into 3x3 pixel and each pixel has 16x16 vector embedding which is flattened to 256x1 (means 256 rows and 1 column). Because we have 3x3 pixel size of image it means we have total 9 pixels. Hence if we combine the vector embedding of all the pixels (means if each pixel embedding is 256x1, then for 9 pixels it will become 256x9 i.e 256 rows and 9 columns. Now we have to pass this through linear layer. Linear layer let's say has 5 neurons. so shape for each neuron will be 256 x 1 (means 256 rows and 1 column) and for 5 neurons it will become 256x5 (menas 256 rows and 5 columns).
Now we have to do matrix Multiplication of Input with Linear layer, but wait, we cannot multiply the matrix because shape of input is 256x9 and shape of linear is 256x5. In order to multiply the matrices, the columns of Matrix A must be equal to the number of rows of Matrix B. So we will transpose the input matrix of shape 256x9 to 9x256.
Now, Let's take this as Matrix A of 9x256 and Matrix B of size 256x5. Because column of Matrix A is same as row of Matrix B, hence, dot product is possible which will result in new matrix of size 9x5.
We can see that the original matrix of patch was of size 9x256 which is reduced to 9x5.
So we will get the 3 matrices of size 9x5 each for Key, Query and Value.
Now based on attention model we can see that we have to do the matrix multiplication of Query and Key and to do so we again have to do the transpose of Matrix because both Key and Query are of same shape (Query Matrix - 9x5 , Key Matrix - 9x5). So if we transpose Key Matrix it will become 5x9 and then the matrix multiplication will be possible between these two matrices (9x5 and 5x9). The dot product output of these matrices will be a matrix of size (9x9) and this output matrix is called as Attention Filter.
Then after training we have the final updated values of this attention filter which we have to scale between 0 and 1 using softmax activation function.
This scaled attention matrix (9x9) is then multiplied with Value matrix (9x5) which will give the filtered value of Matrix (9x5).
Hence based on attention matrix we get the important feature of an image. This is the process of single attention head to extract feature.
We use multi-head attention to extract various important features of an image. Each head focuses on different combinations of features.
Excellent explanation
Glad it was helpful!
Thank you soo much mam for this amazing video
Most welcome 😊
Can you please suggest how to use vision transformer for Text classification? Please
Thanks for making the video
My pleasure!
Great Content!!
Thank you!
Thanks for this tutorials its simple and deep
You're welcome 😊
Awesome explanation mam
Glad you liked it
Please can you explain or give a series about the Vanilla Vision transformers from the paper to the to the programming side🙏🙏
The terms "Vanilla Vision Transformers" and "Vision Transformers" are often used interchangeably, and both refer to the same fundamental concept which is applying the Transformer architecture directly to image data for computer vision tasks.
can you make a video on SegFormer? thanks in advance for the amazing explanation!
I will try!
Nicely Explained..!
Thank you
the video was awesome . And can i know the transformer model of all the 6 encoders and 6 decoders , as I have confusion in the input architecture of decoders .
Thank you mam
Kindly post a video for Deit
Noted!
how to know the feature importance which are generated from ViT ?
which features causes classification
While ViT doesn't inherently provide feature importance scores like some other models, you can analyze the importance of different features in the classification process by examining the attention maps generated by the model.
Attention maps in ViT represent the importance of each image patch in relation to the final prediction. Higher attention values indicate greater importance. By visualizing these attention maps, you can gain insights into which image regions contribute most to the classification decision.
@@CodeWithAarohi please make video on it madam, for one classification task , dog vs cat classification example
excellent explanation.
I wanna make a sugesstion. Maybe you should buy a microphone. There are lots of noise in background.
Thank you, I will
very nice video.
Many many thanks
Good one ma’am
Thanks a lot
can i get the ppt?
please create a ViT on landmark detection
I will try
@@CodeWithAarohi thank you so much you are the best
Waiting for your fusion transformer tutorial mam
Ok sure
Thanks a lot! it helps me :3
I'm glad!
Code with Aarohi is Best RUclips channel for Artificial Intelligence #CodeWithAarohi
Can these be apply in bank cheque processing
Yes, vision transformers (ViTs) can be applied to bank cheque processing tasks.
Thank you for making this video. Please make a python code of ViT, if possible. Thank you.
Working on it!
Please, do it on how to train ViT on semantic segmentation tasks.
why we we are using 16x16 patches
Because in original paper of ViT, they have used patch size 16*16. You can work with some other patch size also.
Eg: Patch Size 8×8 - which will give you 784 patches if your image size is 224*224. Another example: Patch Size 32×32 will give you 49 patches if image size is 224*224
thank you so much Aarohi, please,could you explain SWIN transformer too with its with coding ?
Sure, I have started a playlist on transformers and will try to cover every important topic which comes under transformers
@@CodeWithAarohi thank you again, waiting for it
I came to this video to learn how to do positional encoding for 2D images -- the precise math. When you come to that portion, you simply reference your intro video, re Transformers for linear text (and in which even the linear positional encoding isn't really explained).
Sorry for inconvenience. I will try to cover the math's in separate video.
@@CodeWithAarohi Thanks for responding. Your videos are terrific otherwise. Thanks for sharing your work and insights.
‘The precise math for encoding’
Bro it’s deep learning and you talk about precise math 😂
Literally those people encoded 1,2,…256 for each patch.
WOW WOW WOW!!!!
Thank you :)
nice, can you share slide with me?
Can you provide code
In this video, I have explained Vision transformer theory. You can check next video and Code link is mentioned in description section of that video.
will you do vision transformers with tensorflow?
I will try.
@@CodeWithAarohi Thank you. Your method of teaching is amazing. But i am never comfortable with torch. Tensorflow is so natural for deep learning. I look forward to this .