If anyone is confused at 2:55 , why he is multiplying by V transpose and not the inverse, to get rid of V on the left side. It's because V is orthogonal matrix (that is how we defined it to be), and the transpose of an orthogonal matrix is equal to its inverse. This stems from the fact that if a matrix Q is orthogonal then QQ_T = Q_TQ = I
@@fernandamacedo4472 Yes, you are correct. He should've said the initial vectors we are using are orthonormal not just orthogonal. Unfortunately math has a lot of very similar words for very similar concepts, and is easy to get confused. For instance, if a matrix has orthogonal but non-unit-length vectors as columns (or rows), it wouldn't satisfy the condition QQ_T = Q_TQ = I. But QQ_T = Q_TQ = D, where D is a diagonal matrix whose diagonal entries represent the squares of the norms of the columns (or rows).
after watching gilbert strang I needed exactly this to visualize and understand very clearly. You taught most difficult to explain concept in Linear algebra , Machine learning and even deep learning in 7 minutes. Really grateful. You are at the level of 3blueandbrown and gilbert stang in terms of explaining difficult concept like its understandable to even beginners. Keep the good work. I would subcribe for paid contents for sure. Sadly very under appreciated channel but hope people will make it popular. Waiting for PCA.
A (rather well-known) mathematician at my graduate institution once said, "The SVD is the single-most beautiful thing that can be said about a matrix". After seeing this, I'm convinced that he's right.
There is a giant error in this video. At 3:07 the matrix A is defined as A=USV_T. Up until 6:43, the proof holds that the arg max will correspond to the vector with the largest corresponding eigenvalue. However at 6:44, the error is that A_T*A is NOT USU_T as A_T*A would be (USV_T)_T * USV_T = V*S*U_T*U*S*V_T. This would mean that the vector corresponding to the maximum stretching would be the first column of V and not U. This is very misleading to viewers.
@@amaygada919 The first column corresponds to the largest singular value. A*v = sigma*u. The vector u is unit length for all columns in the matrix U (V is also unit length for all column vectors in the matrix). This means that A*v1 will correspond to the output of greatest magnitude since sigma1 is the largest.
@@evanparshall1323 why the sigma 1 has to be the largest? We did not assume any rule about that at the beginning? What makes sigma 1 bigger than sigma 2 for instance? (Btw thanks for the giant error correcting, that was so important I guess)
You're right, and I also think that it's important to emphasize that Σ contains all of the square roots of the eigenvalues on its diagonal in descending order. (i.e at 6:43, in addition to the U's needing to be switched to V's for the equation defining (A^T)*A ,the Σ should actually be Σ^2)
Very nice! But, in the beginning, A= USV^T so AA^T = US^2U^T, but then towards the end, you say A^TA = US^2U^T. I guess you switched the notation V U ?
I really like your video. It is minimalistic and helps visualizing so well. I see it's been almost 1 year and there is no othe video from you. Please come back. Eagerly waiting for your next video.
This ia great. Well explained. In your next video you could also talk about compressing high dimensional data by truncating the small-eigenvalue terms in the SVD. It's an important step in tensor network simulations etc.
You have a typo ("eigne" instead of "eigen") at 3:44 and 4:17. Otherwise, great video, thank you very much! It's the first video I watch from you, but if the others are of similar quality, I'm really suprised your channel hasn't blown up yet considering this video is already three years old.
so.. the right singular vectors are the ONB for the rows and the left singular vectors are ONB for the image of A multiplied by its right singular vectors. Geometric interpretation is somewhat intuitive but what do they mean?
nice video. last part doesn't make so much sense tho. all you seem to show is that x^tA^tAx attains the eigenvalues of A^tA when x= the eigenvectors of A^tA. why is one of those the max of x^tA^tAx where x ranges over all unit vectors? rest of video is great though. great visualizations.
Great video, very simple and clear. Just a question: I can’t understand why the columns of U are the eigenvectors of A^T A. Could you please explain this? Thanks
well explained, but i don't agree with your statement about u_1 having the largest eigenvalue. after all, you could just switch v_1 and v_2 and u_1 and u_2, and then the second column has the largest eigenvalue
Thank you, I think I now understand the bigger picture of what SVD actually is, beyond just the formula. You remind me very much of 3B1B.
3B1B's software Manim (Math animator) is free (open-source) for anyone to use.
If anyone is confused at 2:55 , why he is multiplying by V transpose and not the inverse, to get rid of V on the left side. It's because V is orthogonal matrix (that is how we defined it to be), and the transpose of an orthogonal matrix is equal to its inverse. This stems from the fact that if a matrix Q is orthogonal then QQ_T = Q_TQ = I
Thank you so much for the clarification
I think he should have mentioned at the beginning that the initial vector is of length one so the set of vectors are orthonormal, right?
@@fernandamacedo4472 Yes, you are correct. He should've said the initial vectors we are using are orthonormal not just orthogonal. Unfortunately math has a lot of very similar words for very similar concepts, and is easy to get confused.
For instance, if a matrix has orthogonal but non-unit-length vectors as columns (or rows), it wouldn't satisfy the condition QQ_T = Q_TQ = I. But QQ_T = Q_TQ = D, where D is a diagonal matrix whose diagonal entries represent the squares of the norms of the columns (or rows).
after watching gilbert strang I needed exactly this to visualize and understand very clearly. You taught most difficult to explain concept in Linear algebra , Machine learning and even deep learning in 7 minutes. Really grateful. You are at the level of 3blueandbrown and gilbert stang in terms of explaining difficult concept like its understandable to even beginners. Keep the good work. I would subcribe for paid contents for sure. Sadly very under appreciated channel but hope people will make it popular. Waiting for PCA.
you explained it better than gilbert strang
Don't compare bro, trust me he is inspired by prof. Strang like many of us...
yep. It is very unfortunate that strang didn't get to mention that the whole point was "orthonormal matrices still after being transformed"
At 2:55, inverse(V) is equal to transpose(V) because V is an orthogonal matrix
Bro, please make this second video relating to PCA, I beg you. Your explanations are the best
I was looking for SVD in 3blue1brown channel. And you created using same library. Thank you
Make the sequel god damn it. I've been waiting for 3 years.
This is the clearest explanation I've been able to find thank you!!
Great video, thank you. Eagerly waiting for your next PCA video.
Where is the next video about PCA, loved this video.
Hi! I was wondering when you would upload the PCA video, your method of explanation was really lucid and enlightening!
A (rather well-known) mathematician at my graduate institution once said, "The SVD is the single-most beautiful thing that can be said about a matrix". After seeing this, I'm convinced that he's right.
Great work, keep it up!
Minor typos at 3:45 (eigendecomposition), 5:21 (maximum).
Great video! Digestible and beautifully animated. This channel can become a good complement to 3b1b.
Wowww when the next part ?!
It’s amaizing
There is a giant error in this video. At 3:07 the matrix A is defined as A=USV_T. Up until 6:43, the proof holds that the arg max will correspond to the vector with the largest corresponding eigenvalue. However at 6:44, the error is that A_T*A is NOT USU_T as A_T*A would be (USV_T)_T * USV_T = V*S*U_T*U*S*V_T. This would mean that the vector corresponding to the maximum stretching would be the first column of V and not U. This is very misleading to viewers.
why does it have to be the first column and not the second?
@@amaygada919 The first column corresponds to the largest singular value. A*v = sigma*u. The vector u is unit length for all columns in the matrix U (V is also unit length for all column vectors in the matrix). This means that A*v1 will correspond to the output of greatest magnitude since sigma1 is the largest.
@@evanparshall1323 why the sigma 1 has to be the largest? We did not assume any rule about that at the beginning? What makes sigma 1 bigger than sigma 2 for instance? (Btw thanks for the giant error correcting, that was so important I guess)
@@bulutosman That is just how the SVD is defined. The largest singular value will always be at the 1,1 entry second largest at 2,2 etc
You're right, and I also think that it's important to emphasize that Σ contains all of the square roots of the eigenvalues on its diagonal in descending order. (i.e at 6:43, in addition to the U's needing to be switched to V's for the equation defining (A^T)*A ,the Σ should actually be Σ^2)
Thanks so much.
This is a whole new perspective for me.
Loved it!!!!
Keep up the amazing work
Manim will help in creating a lot of amazing content.
Eagerly waiting for your next video on PCA. Please upload soon. This video was brilliant.🤩💥💥
Very nice! But, in the beginning, A= USV^T so AA^T = US^2U^T, but then towards the end, you say A^TA = US^2U^T. I guess you switched the notation V U ?
I really like your video. It is minimalistic and helps visualizing so well. I see it's been almost 1 year and there is no othe video from you. Please come back. Eagerly waiting for your next video.
This ia great. Well explained. In your next video you could also talk about compressing high dimensional data by truncating the small-eigenvalue terms in the SVD. It's an important step in tensor network simulations etc.
Great explanation , and also I couldn't find the next video on Principal Components.
You did a really good job with this, very very clear. Hope you find time to get back to doing this.
Great video but I'm a bit confused at 6.10, at what point is it proven that the maximum is obtained for an Eigenvector of A^TA
Best explaination video on SVD ever
A video that really helps!!! Thanks, buddy please continue to make videos!
Love from India!
Very sightful and I really want to see the explaination of PCA in the next video
What an amazing explanation of SVD.
I'm looking forward to watching your next video sir 😊
why does this not have a million views or at least a couple hundred thousand...
I only have one word in reaction to your video: wow
This so well done man
Thank you very much. Please make more of these.
epic work brotha
Very helpful!!!!!!!!! You make everything so clear
You saved my life man. Thank you.
one of the best videos I have ever seen. keep up the good work
@6:45 eigen vectors of A^TA is column of matrix V right and not U as he said,right? What am I missing
Thanks! So well explained
Excellent video! More please!!
Bro dropped a banger and disappeared forever
Great explanation.
Great Video. Waiting for the PCA
wow! so fluent explanation
Which software do you use for these animations. They are great!!
You make it very clear
This is really insightfully presented.
wonderful video, very clear, intuitive.
Thank you so much, I didn't get it in my uni lecture but got it thanks to you now :)
Amazing video. Thank you very much!
Awesome Video! Thank you.
You have a typo ("eigne" instead of "eigen") at 3:44 and 4:17.
Otherwise, great video, thank you very much! It's the first video I watch from you, but if the others are of similar quality, I'm really suprised your channel hasn't blown up yet considering this video is already three years old.
This was great! Please do more!
6:10 why ATAx can be written as lambda x? What if x is not the eigenvector of ATA?
i dont understand either，could you explain it to me？
so.. the right singular vectors are the ONB for the rows and the left singular vectors are ONB for the image of A multiplied by its right singular vectors.
Geometric interpretation is somewhat intuitive but what do they mean?
this video is amazing, it helped me so much, can't wait to see the next one ;)
Best video on svd
How does one come up with such insightful way to explain things? What steps are needed?
Thanks for wonderful work! At 6.57, I guess you wanted to say V is an eigen vector matrix for A' A.
Excellent vid
Great! When is the next?
2:21 The right hand side of 3rd equation you write u1, u2 (unit vectors) as columns. Doesn't we write them as rows.......... ❓ ❓ ❓
It is noted that "V is an orthogonal matrix because A_T A is symmetric", How we say that ?
Can't wait for PCA ❤️❤️❤️❤️
please make the video on principal component analysis!
Where's the next part, the SVD and PCA?
Awesome video! Look forward to the next one :D
thanks for geometric animations
Excellent. Thank you so much.
nice video. last part doesn't make so much sense tho. all you seem to show is that x^tA^tAx attains the eigenvalues of A^tA when x= the eigenvectors of A^tA. why is one of those the max of x^tA^tAx where x ranges over all unit vectors? rest of video is great though. great visualizations.
Great video! Sadly, where is next one?
This is such a inspiring video! It shows transformation so clear! I got the intuition. Thank you so much! Subscribed : )
Where is the next video? I hope you are all right, given covid and all...
You seem to assume that the solution to the argmax of x^T(A^TA)x is an eigenvector. What's the explanation?
hi，could you explain it to me if you have understand it？
Thanks, very helpful
This gives a really good intuition. The only thing missing is the generalization to non-square matrices
nice visualization, keep going
Pretty cool my man
Great video!
Please post your second video!
Did you base this on Gilbert Strangs lecture?
Beautiful
This was realy helping
Thank you.
Thank you very much
What kind of a matrix has vectors as its elements? Isn't that a tensor?
Nice video
Nice video but i didn't properly understand what he did at 2:06 . Can someone please explain it to me?
masterpiece genius !!
But why are we asked to find an orthogonal set of vectors that when transformed by A will remain orthogonal !
Great video, very simple and clear. Just a question: I can’t understand why the columns of U are the eigenvectors of A^T A. Could you please explain this? Thanks
That is because A^TA is a symmetric matrix and can be decomposed using its Eigen Vectors in the form of Q(lambda)Q^T which would be U in this case
well explained, but i don't agree with your statement about u_1 having the largest eigenvalue. after all, you could just switch v_1 and v_2 and u_1 and u_2, and then the second column has the largest eigenvalue
thank you
Thank u!
At 6:00', how does ||Ax||^2 become (Ax)^T(Ax)??
TY!!!
where is the 2 video
PLEASE UPLOAD MORE
I wish this same very good video existed without the background music. Really interrupting
The next video about SVD in PCA will never be out XD
amazing