Visual Explanation of Principal Component Analysis, Covariance, SVD

Linearity I, Olin College of Engineering, Spring 2018
I will touch on eigenvalues, eigenvectors, covariance, variance, covariance matrices, principal component analysis process and interpretation, and singular value decomposition.

[@zacmac]

Great clarity. You clearly understand your stuff from a deep level so it's easy to teach.

[@user-or7ji5hv8y]

Wow, that was quite good explanation.

[@ramielkady938]

PS: Video is targeted at people who already have a deep knowledge of what the video is trying to explain.

[@riazali-vi8tu]

Well explained, you should do more videos

[@MathematicsMadeSimple1]

Clear explanation. Thank you for shading more light especially on the application of eigenvalues and vectors.

[@patyyyou]

Nicely done. It hit the right level for someone who understands the linear algebra behind Eigenvectors and Eigenvalues but still needed to make the leap of connecting a dot or two in the application of PCA to a problem. Again, thank you!

[@anuraratnasiri5516]

Beautifully explained! Thank you so much!

[@Darkev77]

I do understand that eigenvalues represent the factor by which the eigenvectors are scaled, but how do they signify “the importance of certain behaviors in a system”, what other information do eigenvalues tell us other than a scaling factor? Also, why do eigenvectors point towards the spread of data?

[@malstroemphi1096]

If you consider a raw matrix or just geometric examples eigenvalues are just a scaling factor indeed. And you cannot say much more. But here, we are talking with additional context: we know we are doing statistics and putting "data" into a covariance matrix, which means we can now add more interpretations. The eigen vector is not just some eigenvector of some matrix, it's the eigenvector of a *covariance matrix* in the context of statistics, we've put data into a matrix whose elements measure all the possible spread of data, which is why we can now say an eigenvector points towards the spread of data and its length (eigenvalue) relates to the importance of that spread.

[@eturkoz]

Your explanations are awesome! Thank you!

[@szilike_10]

Believe it or not, I've been wondering a lot about the concept of covariance because every video seems to miss the reason behind the idea. But I think I kind of figured it out today before watching this video and I drew the same exact thing that is in the thumbnail. So I guess was thinking correctly : ))

[@apesnajnin]

Really, amazing lecture! It's make my conception clear regarding eigenvalue and eigenvector. Thanks a lot!

[@spyhunter0066]

Around the minute of 1.36, you said "we divide by n for covariance", but we divide by n-1, instead. Please, do check on that. Thanks for the video. Maybe, I sohuld say estimated covariance has the n-1 division.

[@TheSyntaxerror1]

Love this video, great work!

[@roshinroy5129]

Awesome explanation!! Nobody did it better!

[@bottom297]

Extremely helpful. Thank you!

[@softpeachhy8967]

1:37 shouldn’t the covariance be divided by (n-1)?

[@danielheckel2755]

Nice visual explanation of covariance!

[@bootyhole]

Excellent video, thank you!

[@davestaggers2981]

Graphical interpretation of covariance is very intuitive and useful for me. Thank you.

[@nickweimer6126]

Great job explaining this

[@vietdaoquoc7629]

thank you for this amazing video

[@simonala7090]

Would love to request an in person version

[@Muuip]

Great concise presentation, much appreciated! 👍

[@skshahid5565]

Why do you stop making videos?

[@123arskas]

Thank you. It was beautiful

[@EdeYOlorDSZs]

poggers explination thankyou

[@ivandda00]

ty

[@Timbochop]

Good job, no wasted time

[@Pedritox0953]

Good explanation

[@Agastya007]

Plz do more videos

[@arjunbemarkar7414]

How do u find eigenvalues and eigenvectors from the covariance matrix?

[@Eta_Carinae__]

Same as usual, right? Find lambda using det(Sigma - lambda * I) = 0, so just take lambda away from the main diagonal of the Cov. Matrix, take the determinant of that and you'd be left with some polynomial of lambda which you then solve for, each solution being a unique eigenvalue.

[@sakkariyaibrahim2650]

Good lecture

[@Trubripes]

Thanks for concisely explaining that PCA is just SVD on the covariance matrix.

[@latanezimbardo7129]

1:28 I personally visualise covariance like this, I always thought i was wrong, I have never seen others doing this, how come??

[@TechLord79]

Very well done!

[@m.y.s4260]

awesome explanation! thx!

[@prof.laurenzwiskott]

Very nice video. I plan to use it for my teaching. What puzzles me a bit is that the PCs you give as an example are not orthogonal to each other.

[@saDikus1]

Great video! Can anyone tell how she decided that PC1 is spine length and PC2 is Body mass? Should we guess (hypothesize) this in real world scenarios?

[@f0xn0v4]

I have always dreaded statistics, but this video made these concepts so simple while connecting it to Linear algebra. Thank you so much ❤

[@liviyabags]

I LOVE YOU !!!!! whattay explanation... thank you so much

[@tusharkush7]

This video needs a golden buzzer.

[@simonala7090]

Agreed!!!

[@blackshadowofmysoul]

Best PCA Visual Explanation! Thank You!!!

[@sebgrootus]

Incredible video. Genuinely exactly what i needed.

[@matato2932]

thank you for this amazing and simple explanation

[@Lapelu9]

I thought PCA was a hard concept. Your video is so great!

[@zendanmoko5005]

Thank you! very nice video, well explained!

[@1291jes]

This is excellent, Emma... I will subscribe to your videos!

[@jordigomeztorreguitart]

Great explication. Thank you.

[@VivekTR]

Hello Emma, Great job! Very nicely explained.

[@skewbinge6157]

thanks for this simple yet very clear explanation

[@getmotivated3619]

You are awesome... u make a mediocre out of a knownothing.

[@Agastya007]

I love the way u spelled "data" at [3:34]😁😁

[@basavg1]

Very Nice..pls keep posting

[@thryce82]

nice job was always kinda confused by this.

[@abdulrahmanmohamed8800]

A very good explanation.

[@subinnair3835]

Dear mam,
How did you obtain the matrix at 5:30 ?

[@emfreedman3905]

Find the Covariance Matrix of these variables, like at 2:15, and find its eigen decomposition (find its two dominant eigenvectors). The matrix at 5:30 is the two dominant eigenvectors. Each column is an eigenvector.

[@subinnair3835]

Emma Freedman thank u !
The video's explanation was great and covered all the fundamentals required to fully understand PCA !! 😃

[@tusharpandey6584]

awesome explanation! make more vids pls

[@vitokonte]

Very nice explanation!

[@stephenaloia6695]

Thank you, Ma'am!

[@tractatusviii7465]

investigate hedge/hogs

[@crispinfoli9448]

Great video, thank you!

[@DanielDa2]

great explanation

[@KimJennie-fl3sg]

I just love the voice🙄😸

[@nbr2737]

beautiful, thanks a lot!

[@Matt-bq9fi]

Great explanation!

[@siliencea9362]

thank you so much!! :)

[@haroldsu]

Thank you for this great lecture.

[@mahmoudreda1083]

thanks A LOT

[@DoFlamingo_1P]

AWESOMEEE 🤘🤘🤘

[@adriantorresnunez]

Best explanation I have heard from PCA. Thank you

[@astiksachan8135]

5:12 was very good

[@콘충이]

Awesome!

[@SpeakerSparkTalks]

nicely explained

[@checkout8352]

Tnks

[@raghav5520]

Well explained

[@astiksachan8135]

4:35

[@AEARArg]

Congratulations Emma, your work is excellent!

[@2894031]

babe var(x,x) makes no sense. either you say var(x) or cov(x,x)

[@ABC-hi3fy]

No one explains why they use covariance matrix. Why not use actual data and find its igen vector/igen values. I have been watching hundreds of videos books. No one explains that. It just doesn't make sense to me to use covariance matrix. Covariance is very useless parameter. It doesn't tell you much at all.

[@malstroemphi1096]

No it does especially using PCA. But you are right, you need actual data. Say the data are 3D points of some 3d objects, if you use this technique (build a cov matrix using the 3D points and do the PCA of it) then you will find a vector aligned with overall direction of the shape: for instance you will find the main axis of a 3d cylinder. This is quite a useful information.