NOTE: The StatQuest LDA Study Guide is available! statquest.gumroad.com Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
Hi Josh, Love your content. Has helped me to learn a lot & grow. You are doing an awesome work. Please continue to do so. Wanted to support you but unfortunately your Paypal link seems to be dysfunctional. Please update it.
the funny thing is, so many materials from this channel are for those university students (like me) but he keeps treating us like kindergarten children. Haha feels like i'll never be growing up, by watching your videos sir! QUADRO BAAM SIR, THIS WORLD HAS BEEN GONE TOO SERIOUS, THANK YOU FOR BRINGING BACK THE JOY
Just spent hours so confused, watching my lectures where the professor used only lin alg and not a single picture. Watched this video and understood it right away. Thank you so much for what you do!
This was honestly helpful, i am an aspiring behavioral geneticist (Aspiring because I am still an undergraduate of biotechnology) with really disrupted fundamentals of math especially statics. Your existence as a youtube channel is a treasure discovery to me !
Amazingly, my professor did not even discuss projecting data to new axes that maximize linear separability of the groupings. Thank you so much for the core intuition so I dig in a little further.
You, sir, you are a life saver. Now in every complicated machine learning topics I look for your explanation, or at least wonder how you would have approached this. Thank you, really.
Awesome! Even I get it and love it! I'm going to share one of your stat-quest posts as an example of why simple explanations in everyday language is far superior to using academic jargon in complex ways to argue a point. Also, it's a great example of how to develop an argument. You've created something here that's useful beyond statistics! Three cheers for the liberal arts education!!!! Three cheers for Stat-Quest!!
@@Sachin-vr4ms I'm sorry that it is confusing, but let me try to explain: At 9:46, imagine rotating the black line a bunch of times, a few degrees at a time, and using the equation shown at 8:55 to calculate a value at each step. The rotation that gives us the largest value (i.e. there is a relatively large distance between the means and a relatively small amount of scatter in both clusters) is the rotation that we select. If we have 3 categories, then we rotate an "x/y-axis" a bunch of times, a few degrees each time, and calculate the distances from the means to the central point and the scatter for each category and then calculate the ratio of the squared means and the scatter. Again, the rotation with the largest value is the one that we will use. Does that help?
The song at the beginning made my day, even though I took wrong tutorial of Linear discriminant analysis in data science. Just awesome. Love it a lot. We need more and more funny teachers like you.
Hi Josh, Helpful to understand the differences between PCA and LDA and how LDA actually works internally. You're indeed making life easier with visual demonstrations for students like me :) God bless and Thank you!
This helped me understand LDA before my midterm! I could not wrap my head around how the functions worked and what they did, but I got an "ah-hah!" moment at 6:49 and I totally understand it now. Thank you for explaining this!
Great video! I initially couldn't understand LDA looking at the math equations elsewhere, but when I came across this video, I was able to understand LDA very well. Thanks for the effort.
Another excellent video just as great as the one on PCA. I read a Professor's view on most of the models and algorithms stuff in ML where he recommended understanding the concepts well so that we know where to apply and not worry too much about the actual computation at that stage. The thing that is great in your videos is that you explain the concept very well.
wow... my professor has been trying to teach me the concepts for weeks. and now I finally understand. Thank you so much. I will refer this to my mates.
Hey Josh, I am really thankful for the videos you are making and posting. I am very much motivated and inclined towards learning machine learning and most of the sources didn't give such a fundamental explanation of how things work.
And, I am not promising but I do really look forward to buying your song and gifting it one of my friend with whom I share the same music taste and who also happens to be an expert in Python
Really great videos, saved me from my data science classes. I'm applying for graduate program at UNC, hope I can have the opportunity to meet the content creators sometime in the future.
10/10 intro song 10/10 explanation using PCA, I can reduce these two ratings to just one: 10/10 is enough to rate the whole video using LDA, the RUclips chapters feature maximizes the separation between these 2 major components (intro and explanation) of the video
Great video! Just wanted to point out that LDA is a classifier, which involves a few more steps than the procedure described here, such as assumption that the data is gaussian. The procedure here described is only the feature extraction/dimensionality reduction phase of the LDA. G
You are correct! I made this video before I was aware that people had adapted LDA for classification. Technically we are describing "Fisher's Linear Discriminant". That said, using LDA for classification is robust to violations to the gaussian assumptions. For more details, see: sebastianraschka.com/Articles/2014_python_lda.html
StatQuest with Josh Starmer That said, I must admit I am having a really hard time understanding how the fisherian and baysian approach lead to the same conclusion even with completely different routes. If you have any source on that it would be of enormous help for my sanity haha
Thanks for the video! I have an exam next week and even though its open book, I still didn't feel comfortable going into it. This video definitely helped!
@@statquest Thank you so much for this video. I tried to understand LDA by reading lots of materials (books, papers, etc.), but none of them can explain things as clear as you do. Really appreciate it!
Amazing. Thank you for this excellent video. Explained everything super clearly to me in a super concise manner without all the academic jargon getting in the way.
"But what if we used data from 10k genes?" "Suddenly, being able to create 2 axes that maximize the separation of three categories is 'super cool'." Well played, StatQuest, well played!
Loved the explanation. Your channel has been a truly invaluable source for studying ML. I was wondering whether you could make a video on the differences/similarities along with use cases for KNN/LDA/PCA.
Hello Josh. As always, thank you for your super intuitive videos. I won't survive college without you. I do have an unanswered conundrum about this video, however. For Linear Discriminant Analysis, shouldn't there be at least as many predictors as the number of clusters? Here's why. Say p=1 and I have 2 clusters. In this case, there is nothing I can do to further optimize the class separations. The points as they are on the line already maximizes the Fisher Criterion(between-class scatter/in-class scatter). While I do not have the second predictor axis to begin with, even if I were to apply a linear transformation on the line to find a new line to re-project the data on, it will only make the means closer together. Extending this reasoning to the 2D case where you used gene x and gene y as predictors and 3 classes, if the 3 classes exist on a 2D plane, there is nothing we can do to further optimize the separation of the means of the 3 classes because re-projecting the points on a new tilted 2D plane will most likely reduce the distances between the means. Now, if each scatter lied perfectly vertically such that as Gene Y goes up the classes are separated distinctly, then we could re-project the points on a new line(that would be parallel to the invisible vertical class separation line) to further minimize each class's scatter, but this kind of case is very rare. Given my reasoning, my intuition is that an implicit assumption for LDA is that there needs to be at least as many predictors as the number of classes to separate. Is my intuition valid?
Hey man! That was a nice clear cut explanation . I have been doing machine learning using LDA but I never knew what this LDA actually does . I only had a vague idea . By the way , you wrote "seperatibility" instead of "separability " at 5:26 ....
@@thenkprajapati Cool! However - just so you know, it could still be awhile before I make the video. I get about 3 requests every day, but I can only make about 2 videos a month. The more people that ask for a specific topic, the more priority I give that topic. So if you know a lot of people interested in Independent Component Analysis or SVD, tell them to put in their requests as well so that I'll prioritize these subjects.
Like PCA, LDA "compress" the data into lower dimensions. But unlike PCA, it do so while keeping/maximising the classificability (separability) of the data according to the given classification, as much as possible. Data must already be classified to use LDA. Find the line (the new lower dimension) such that the difference between the means of two classes of data are maximised. At the same time, the variance among the data of the same class is minimised.
NOTE: The StatQuest LDA Study Guide is available! statquest.gumroad.com
Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
woohoo
Can you please do something on canonical analysis ?
@@WIFI-nf4tg I'll keep that in mind.
Hi Josh,
Love your content. Has helped me to learn a lot & grow. You are doing an awesome work. Please continue to do so.
Wanted to support you but unfortunately your Paypal link seems to be dysfunctional. Please update it.
website shows "Error establishing a database connection" !
the funny thing is, so many materials from this channel are for those university students (like me) but he keeps treating us like kindergarten children. Haha feels like i'll never be growing up, by watching your videos sir! QUADRO BAAM SIR, THIS WORLD HAS BEEN GONE TOO SERIOUS, THANK YOU FOR BRINGING BACK THE JOY
Thank you very much! :)
I am a kindergarden kid in this subject : (
@@daisy-fb5jc same here; i need someone to explain it like im a little kid
Remember: Us Adults are just big children
Every time I heard the intro music. I know my assignment is due in 2 days.
LOL! :)
@@statquest Thank you very much!
hahahah I'm on the same boat right now
Good to know I'm not alone.
10,5 hours till my machine learning exam. Thank you so much, I feel way better prepared than if I would have watched all of my class material.
Just spent hours so confused, watching my lectures where the professor used only lin alg and not a single picture. Watched this video and understood it right away. Thank you so much for what you do!
Glad it helped!
This is amazing! 15 mins video does way better than my lecturer in an 2 hours class
While these 15 min videos are excellent for gaining intuition, you still often need those 2-hour classes to get familiar with the mathematical rigor.
@@elise3455 no you dont. math follows super quick and easy when you understood what it is about
@@NoahElRhandour yeah brother
@@elise3455 No you don't. Math become super easy once you understand what you doing
This was honestly helpful, i am an aspiring behavioral geneticist (Aspiring because I am still an undergraduate of biotechnology) with really disrupted fundamentals of math especially statics. Your existence as a youtube channel is a treasure discovery to me !
Thanks! :)
Amazingly, my professor did not even discuss projecting data to new axes that maximize linear separability of the groupings. Thank you so much for the core intuition so I dig in a little further.
Glad I could help!
Hey what is the intro track called? I couldn't find it on Spotify. . . :D
It's their own
Listen carefully it's the channel name in it and is cool 😂😂👌
@@hiteshjoshi3061 I think they know that and it was a joke^^
You, sir, you are a life saver. Now in every complicated machine learning topics I look for your explanation, or at least wonder how you would have approached this. Thank you, really.
Awesome! Thank you! :)
I really like the systematic way you approach each topic and anticipate all the questions a student might have.
Awesome! Even I get it and love it! I'm going to share one of your stat-quest posts as an example of why simple explanations in everyday language is far superior to using academic jargon in complex ways to argue a point. Also, it's a great example of how to develop an argument. You've created something here that's useful beyond statistics! Three cheers for the liberal arts education!!!! Three cheers for Stat-Quest!!
Are you somehow related to Joshua? :-P
@@rachelstarmer5073 ha
Another great video. Thank you so much. You are definitely one of the best educators in the world.
Wow, thank you!
Wow , that is one of the best explanations of LDA
it helped me get an intuitive idea about LDA and what it actually does in classification
Thank You!
Hooray! Thank you! :)
Can you make a video on quadratic discriminant Analysis
@@Sachin-vr4ms Which part? Can you specify minutes and seconds in the video?
@@Sachin-vr4ms I'm sorry that it is confusing, but let me try to explain: At 9:46, imagine rotating the black line a bunch of times, a few degrees at a time, and using the equation shown at 8:55 to calculate a value at each step. The rotation that gives us the largest value (i.e. there is a relatively large distance between the means and a relatively small amount of scatter in both clusters) is the rotation that we select. If we have 3 categories, then we rotate an "x/y-axis" a bunch of times, a few degrees each time, and calculate the distances from the means to the central point and the scatter for each category and then calculate the ratio of the squared means and the scatter. Again, the rotation with the largest value is the one that we will use. Does that help?
@@Sachin-vr4ms I'm glad it was helpful, and I'll try to include more "how to do this in R and python" videos.
You are just superb!! 8yrs. & still so concise and best explanation
Thanks!
This explains the beauty of LDA so well! Thank you so much!
Awesome! Thank you very much! :)
The song at the beginning made my day, even though I took wrong tutorial of Linear discriminant analysis in data science. Just awesome. Love it a lot. We need more and more funny teachers like you.
Thanks!
Hi Josh, Helpful to understand the differences between PCA and LDA and how LDA actually works internally. You're indeed making life easier with visual demonstrations for students like me :) God bless and Thank you!
Glad it was helpful!
I just graduated from high school, but your videos helped me understand many research papers. Thank you very much!!!!!
BAM and congratulations!!! :)
@@statquest DOUBLE BAM !!
This helped me understand LDA before my midterm! I could not wrap my head around how the functions worked and what they did, but I got an "ah-hah!" moment at 6:49 and I totally understand it now. Thank you for explaining this!
Hooray! Good luck with your midterm. :)
Great video! I initially couldn't understand LDA looking at the math equations elsewhere, but when I came across this video, I was able to understand LDA very well. Thanks for the effort.
Josh. you are an amazing teacher. i have learned so much from you , a big thank you from the bottom ofmy heart. god bless you
My pleasure!
I am able to grasp on this topic without being scared. Kudos to this channel
Thank you!
I didn't understand what the professor talked about in the lecture until I watched your videos. Thanks Josh, you save me!
Happy to help!
woww........toooo goodddddddddddd.....dear Starmer...nothing to say..you are incredible...I am eagerly waiting for your next video...
Never thought anyone could explain things this easily. I appreciate the effort. Thank You
Thank you! :)
Brilliant video! Very helpful. Thank you.
Another excellent video just as great as the one on PCA. I read a Professor's view on most of the models and algorithms stuff in ML where he recommended understanding the concepts well so that we know where to apply and not worry too much about the actual computation at that stage. The thing that is great in your videos is that you explain the concept very well.
Thank you very much! :)
Came for my midterm tomorrow, stayed for the intro track.
Thank you so much for helping me provide a faster solution for the confusion that has taken control of my head for 72h.
Happy to help!
Absolutely brilliant. Kudo's to you for making seem it so simple. Thanks!
much better than my university lecture that I listened to twice but couldn't understand ... this was awesome, thanks!
Hooray! I'm glad the video was helpful. :)
Thank you, very educative and entertaining!
You're welcome! :)
among the best best 15 minutes you can spend on youtube! thank you.
Wow, thanks!
Awesome, just I can say bravo man, bravo, thank you very much.
Thanks!
wow... my professor has been trying to teach me the concepts for weeks. and now I finally understand. Thank you so much. I will refer this to my mates.
When's the StatQuest album coming out? (Here come the Grammies!)
🎸👑
Actually, the only reason I watch your videos is for the music.
😍🎶🎵
You are about to be the reason I pass my qualifying exam in bioinformatics 🙏🙏
Good luck!!! BAM! :)
fact: none of you skipped the intro
This is one of my favorites. :)
I really can't thank you enough for that...you did in 16 mins what I couldn't do in 4 hours. keep on the good work!! and thank you again !!!
Thanks!
4:14 was waiting for the "sound"
:)
Hey Josh, I am really thankful for the videos you are making and posting. I am very much motivated and inclined towards learning machine learning and most of the sources didn't give such a fundamental explanation of how things work.
And, I am not promising but I do really look forward to buying your song and gifting it one of my friend with whom I share the same music taste and who also happens to be an expert in Python
@@bonleofen6722 You're welcome!!! I'm really happy to hear that you like my videos and they are helping you.
They are helping me loads.
Really great videos, saved me from my data science classes. I'm applying for graduate program at UNC, hope I can have the opportunity to meet the content creators sometime in the future.
Best of luck!
"Dr, those cancer pills just make me feel worse"
presses red button "wohp waaaaaaaa"
"next patient please"
:)
You are my hero. I am a senior hoping to get into data science and your videos are great and very helpful. Keep up the good work.
This guy is amazing.
Thanks!
Thanks for this brilliant video! One thing I think is worth mentioning or emphasizing is LDA is supervised and PCA is unsupervised.
Noted
I really liked how you compared the processes of PCA and LDA analysis. I got to know a different way to view LDA due to this video
Bam!
Why is he always singing at the beginning of the video?? Lolol
Can't stop, won't stop! ;)
honestly im not complaining..it shows he is funny and is true to his self :)
Ganesh Kumar Thank you!!
Awesome! It'll be good to give some differences of PCA and LDA. For example, PCA is studying the X. LDA is studying the X->Y.
Nice singing
10/10 intro song
10/10 explanation
using PCA, I can reduce these two ratings to just one: 10/10 is enough to rate the whole video
using LDA, the RUclips chapters feature maximizes the separation between these 2 major components (intro and explanation) of the video
BAM!!! :)
i recommended all your videos to my fellow students in the data analysis course
Thank you very much! :)
I am so glad this channel has grown to around 316k subscribers. Very well explained. The best of bests.
Wow, thank you!
thank you for your kind, slow, and detailed explanation😭
You’re welcome 😊!
Great video! Just wanted to point out that LDA is a classifier, which involves a few more steps than the procedure described here, such as assumption that the data is gaussian. The procedure here described is only the feature extraction/dimensionality reduction phase of the LDA. G
You are correct! I made this video before I was aware that people had adapted LDA for classification. Technically we are describing "Fisher's Linear Discriminant". That said, using LDA for classification is robust to violations to the gaussian assumptions. For more details, see: sebastianraschka.com/Articles/2014_python_lda.html
StatQuest with Josh Starmer That said, I must admit I am having a really hard time understanding how the fisherian and baysian approach lead to the same conclusion even with completely different routes. If you have any source on that it would be of enormous help for my sanity haha
Thanks for the video! I have an exam next week and even though its open book, I still didn't feel comfortable going into it. This video definitely helped!
Good luck and let me know how it goes. :)
Too much time and effort spent, but they worth it. Best explanation I watched after six weeks of search. Cordially thank you.
Thanks! :)
always excited when i look for a topic and its available on statquest
Awesome! :)
very clearly explained. the video is very enjoyable to watch too! Statquest has all that is needed to learn machine learning algos and stats well
Thank you!
I love your stuff, you have the knack to explain things better than most!
Thank you!
@@statquest Thank you so much for this video. I tried to understand LDA by reading lots of materials (books, papers, etc.), but none of them can explain things as clear as you do. Really appreciate it!
@@meng-laiyin2198 Thanks! :)
Best explanation iv ever seen on ML. This is the first time iv watch ML youtube video without rewind :| ..
Keep Up bro..
Wow, thanks!
Amazing. Thank you for this excellent video. Explained everything super clearly to me in a super concise manner without all the academic jargon getting in the way.
Glad it was helpful!
the song in the introduction is always awesome. thanks lol! and very useful video
Thanks!
All of your StatQuest videos are awesome! Thanks for using your time to help others! Much appreciated!
Excellent! You are a better teacher than many overrated professors out there :)
Thank you! :)
I'm more aligned to hear and love the song than the lecture these days :)
:)
You are awesome.Eventually,I was able to reach understanding point of machine learning staffs thanks to you.
Awesome! :)
I get it! You sir is the best lecturer in statistics
Thanks!
That helped me a lot! Thank you sooo much! Now I'm ready for my exam tomorrow :)
Best of luck!
This channel deserves millions of subscribers !!!!
Thank you!
you are very cool bro. I aced my work at my research institute because of youuuuuuuu
That's awesome!!! So glad to hear the videos are helpful. :)
Besides this wonderful explanation, Your music is very good !
Many thanks!
subscribed just because the way you described this topic is so simple and understandable. nice job!
Thank you very much! :)
I wish I can throw this video to my professor, and teach her how to give understandable lectures.
Just a wish.
:)
"But what if we used data from 10k genes?"
"Suddenly, being able to create 2 axes that maximize the separation of three categories is 'super cool'."
Well played, StatQuest, well played!
Thanks!
Tomorrow is my exam, that might be helpful
Thanks a lot from India
It was a very helpful video. I get to understand it in the first attempt only. Thanks a lot for this video sir.
Hooray!!! I'm glad the video was so helpful! :)
Loved the explanation. Your channel has been a truly invaluable source for studying ML. I was wondering whether you could make a video on the differences/similarities along with use cases for KNN/LDA/PCA.
I'll keep that in mind.
great video, you make all the academic terms very understandable, cheers from China
Very useful and intuitive, also sick intro music right there as usual! xD
I think this might be my favorite intro.
This channel is pure gold!
Thank you! :)
Hello Josh. As always, thank you for your super intuitive videos. I won't survive college without you.
I do have an unanswered conundrum about this video, however. For Linear Discriminant Analysis, shouldn't there be at least as many predictors as the number of clusters? Here's why. Say p=1 and I have 2 clusters. In this case, there is nothing I can do to further optimize the class separations. The points as they are on the line already maximizes the Fisher Criterion(between-class scatter/in-class scatter). While I do not have the second predictor axis to begin with, even if I were to apply a linear transformation on the line to find a new line to re-project the data on, it will only make the means closer together. Extending this reasoning to the 2D case where you used gene x and gene y as predictors and 3 classes, if the 3 classes exist on a 2D plane, there is nothing we can do to further optimize the separation of the means of the 3 classes because re-projecting the points on a new tilted 2D plane will most likely reduce the distances between the means. Now, if each scatter lied perfectly vertically such that as Gene Y goes up the classes are separated distinctly, then we could re-project the points on a new line(that would be parallel to the invisible vertical class separation line) to further minimize each class's scatter, but this kind of case is very rare.
Given my reasoning, my intuition is that an implicit assumption for LDA is that there needs to be at least as many predictors as the number of classes to separate. Is my intuition valid?
I believe your question might be answered in this video on PCA tips: ruclips.net/video/oRvgq966yZg/видео.html
I just watched all your videos for intro track :P ......awesome tracks and nicely explained videos
Awesome! :)
Amazing! I subscribed after watching your video only twice!
Wow, thanks!
Another clearly explained video by StatQuest!
BAM! :)
Thankyou , Explanation of LDA & PCA is very clear....
Simply superb! Awesome Josh!!!!
Thank you very much! :)
Great video Joshua ! Looking forward to learning more from you !
Cheers from Japan !
Wow!
At first "wt.f is Statquest"
then
At the end of video, STATQUEST! and I checked on the description. Its a great website !
Thanks
this videos are incredible, i would pay for it if i had money
Sometimes it's the thought that counts! I'm glad you enjoy the videos. :)
Hey man! That was a nice clear cut explanation . I have been doing machine learning using LDA but I never knew what this LDA actually does . I only had a vague idea . By the way , you wrote "seperatibility" instead of "separability " at 5:26 ....
That's embarrassing. One day when StatQuest is making the big bucks I will hire an editor and my poor spelling will no be source of great shame.
Once again, a fantastic job. Thanks, StatQuest.
Thanks again!
I really like your channel, the explanation of concepts was clear and precise!!
Thank you!
Thanks as every time! The best explanations of complicated things!
Thanks!
You're an excellent teacher. Thank you so much.
Thank you! :)
The best explanation on whole internet 💯
Thank you! :)
Indeed clearly explained. Please also make videos on Independent Component Analysis and Singular Value Decomposition.
OK. I'll put those on the to-do list.
@@statquest Thanks. People like me are waiting.
@@thenkprajapati Cool! However - just so you know, it could still be awhile before I make the video. I get about 3 requests every day, but I can only make about 2 videos a month. The more people that ask for a specific topic, the more priority I give that topic. So if you know a lot of people interested in Independent Component Analysis or SVD, tell them to put in their requests as well so that I'll prioritize these subjects.
Very nicely explained! Thank you very much Josh!
Hooray!!! I'm glad you like the video. :)
Tripple bam!!😁🤷
Thank you for the illustration, it's very clear!
Glad it was helpful!
Can i say that the most brilliant thing about statquest is the silly song??? love it, super fan
Like PCA, LDA "compress" the data into lower dimensions. But unlike PCA, it do so while keeping/maximising the classificability (separability) of the data according to the given classification, as much as possible.
Data must already be classified to use LDA.
Find the line (the new lower dimension) such that the difference between the means of two classes of data are maximised. At the same time, the variance among the data of the same class is minimised.
:)