Hello Josh, Could you tell me when and where I should use KNN, KMC, Hierarchical, or other unsupervised machine learnings? By this I mean, are there any metrics to judge which one is better? Or in which situation, this one is more suitable than another one?
In this example, we're using k-nearest neighbors for classification. For example, we might ask if some new person likes the movie Troll 2 or not. In that case, taking the average doesn't make any sense. However, if we were trying to solve some sort of regression problem, like predict how tall some might be, it might make sense to take the average of the k-nearest neighbors.
I love this guy's shtick. Corny, slightly annoying music, although I'm sure he is a great musician. Slightly condescending voice when he goes over the material... like "I'm making this so fucking easy for you... you can't possibly not understand this". It's actually quite calming. He speaks slowly too. You don't have to constantly pause his videos. I understand everyone of his videos. If I don't, it's because I didn't yet watch any prerequisite videos that he tells you at the beginning to watch. He never takes for granted that you understand some detail. This is the BIGGEST freakin' mistake of educators. Some damn variable in a formula that they forget to explain. Also, he will use the simplest example possible so that you understand. I am returning to school, grad school in the ML track for computer science. I don't remember much of the math that I took 20 years ago. This guy is a lifesaver. Wish I watched these when I started. I will be watching all of his videos. After I graduate and make some money, I'm sending him some bucks thru Patreon. Thanks man!
BAM! Thank you very much! I think I must have "resting condescending voice" - because several people have made the comment that I sound a little condescending - but trust me this is not intentional! :)
@@statquest It's actually reassuring. You know, when you are talking to someone who is freaking out? And you make it sound like "Dood, this not that hard."
Just wow thanks Josh. You are just great. One doubt however, if k values are large will outliers not affect my algo? Effect of outliers in knn? Please answer.
These videos are just amazing and clearly are extremely successful in simplifying topics that are usually thought of as difficult. Can you please also make videos on its code in python/R..? and of naive bayes too maybe. That would be super useful. Thank you very much for this level of awesome content.
Your video is amazing as always... It would be great if you can include how to choose the value for 'k' and evaluation metrics for kNN. Also, if I understand it right, there is no actual "training" happening in kNN. It is about arranging the points on the cartesian plane and when a new data point comes, it will again be placed on the same plane and depending on the value of "k", it will be classified. Correct me if I'm wrong.
Hi. Yes, you are right. KNN is easy to implement and understand and has been widely used in academia and industry for decades. You may utilise the cross-validation technique and the validation datasets to select the value for k.
Hello Josh, can you please explain why 3 nearest neighbours are Orange and 1 nearest neighbour is green (the red one looks closest to the black spot to me)? I might have misunderstood the meaning of k nearest neighbours, though PS: loved your explanation, thank you!
When we have categorical variable like Yes/No or type of job (which can take four values: business, healthcare, engineering, or education), how can we calculate distances? Is knn useful at all?
If there is a distance metric, then it KNN will work, and there are distance metrics for categorical variables. See stackoverflow.com/questions/2981743/ways-to-calculate-similarity/2983763#2983763
You can cite StatQuest the same way you would cite any other website or RUclips video. Here is an example: StatQuest. “R-squared explained” RUclips, Joshua Starmer, 3 Feb. 2015, ruclips.net/video/2AQKmw14mHM/видео.html.
Thanks for the very informative info ! Though I have a question , if my dataset is filled with just categorical string data. So no numerical data . Is there a way I can still use knn to predict ? I heard about encoding the string to numerical value but that seems very complex with big dataset .
If you use R, then you can use a Random Forest to cluster anything and then apply KNN to that clustering: ruclips.net/video/sQ870aTKqiM/видео.html If you don't use R, you can use target encoding: ruclips.net/video/589nCGeWG1w/видео.html
Hi . Thanks for video . About the concept of KNN , how the location of unknown cell change in scatter plot . You must change the location of that? And second question, we should change the value of k to reach best k ?
The location of the "unknown cell" is fixed - it does not change. Just the classification changes. I offer a few thoughts about how to pick 'k' at 4:12, but, other than that, you can use cross validation: ruclips.net/video/fSytzGwwBVw/видео.html
Explanation was very very very good. Easily understandable by anyone almost. Can you please do a video on KFold and StratifiedKFold with an example using python. Also, can you explain cross_val_score in details
I have a video on cross validation that covers the concepts in K-Fold cross validation. That might not be exactly what you are looking for, but just in case, here is the link: ruclips.net/video/fSytzGwwBVw/видео.html
I don't understand the purpose of using PCA. Since we have a dataset of known categories, why can't we directly calculate the Euclidean distance between samples of unknown categories and samples of known categories and determine the category to which they belong, like K-means. This sounds stupid but I'm actually a little confused.
@@statquest It went well, thank you! Hopefully I get good grades. I was thinking of suggesting that it would be great if you could cover Markov Chain Monte Carlo and related topics. Thank you again! Your channel has been incredibly helpful!
00:10 K-nearest neighbors is a simple algorithm for classifying data. 00:50 Clustering data using PCA and classifying new cell type 01:29 K-nearest neighbors classifies new data based on nearest annotated cells. 02:12 K-nearest neighbors algorithm assigns a category based on the majority of nearest neighbors' votes. 02:59 K-nearest neighbors algorithm classifies unknown points based on nearest neighbors 03:40 K-nearest neighbors can avoid ties by using an odd K value. 04:22 Choosing the best value for K is crucial for K-nearest neighbors. 05:01 Categories with few samples are outvoted
Low values of K(k=1 or K=2) can be noisy. But in your example, the cells are evenly space. K=1 seems to be perfect or do not have outliers. Or do you mean that in real cases, there is a cluster and not evenly spread like yours?
sir can we replace NaN value of column by mean in such a way that if other parameter value is in a particular range than find the mean and replace . Example..if column BMI has NaN value then if age of that person is 45 then we first find the mean BMI of people with a age of range 40 to 50 and replace with this.Similarly,for other person have NaN BMI ... then first check the age of that person and set an interval age and find mean and replace...
That is awsom how you explain this topics. One suggestion, you could show how the 7 nearest ist red, 3 nearest ist orange and 1 nearest is green for the point in the middle. By my eyes, the 1 nearest neigbour ist still red! and it makes me confuse what does nearest means actually :)
Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
Five minutes explains better than some teachers spent one hour. :)
Thank you! :)
Better than teacher spending semester for me
hahahahaha
@@free_thinker4958 wtf really? also my teacher took 5 minutes that's why I understood nothing
For real, this channel is a godsend.
Could you create the video with Edited Nearest Neighbors and Condensed Nearest Neighbors? Thank you.
I'll keep that in mind.
any video on Naive Bayes algorithm?
Not yet, but it is on the to-do list.
Well I appeared for the stat test today. Ur videos on LDA and KNN helped a lot. But ya I wish even naive Bayes was available!
Hello Josh, Could you tell me when and where I should use KNN, KMC, Hierarchical, or other unsupervised machine learnings? By this I mean, are there any metrics to judge which one is better? Or in which situation, this one is more suitable than another one?
It depends on the field and your goal. Often heatmaps are clustered with hierarchal clustering. PCA is often combined with KNN.
I have read somewhere that you take the k nearest points then take their average? So which one is correct? or in which situation?
In this example, we're using k-nearest neighbors for classification. For example, we might ask if some new person likes the movie Troll 2 or not. In that case, taking the average doesn't make any sense. However, if we were trying to solve some sort of regression problem, like predict how tall some might be, it might make sense to take the average of the k-nearest neighbors.
Is it a good strategy to choose K as the size of the smallest category? so it doesn't out-vote a category with a small amount of samples?
It depends, sometimes k should be smaller.
What if the labels are not binary and you do 3nn and the three neighbours are all diff?
Then you would call it a tie and fail to classify the new data.
Whenever I search for a video tutorial, and you pop up in the search results, my heart fills with joy!!! ^^
Thank you once again!
Hooray!!!!! :)
same here ..not started the video yet but only 1 video on knn .....dont know if i can understand very very well like linear regression
I'm taking a machine learning course at university, and I've been blessed with having found your channel. Keep up the great content!
Hooray! I'm glad the videos are helpful. :)
INTRO IS LEGENDARY BRO : )
Yup, that's a good one. :)
Every time I see your videos I'm simply amazed how you manage to make things simple,it's like 1+1=2, respect
Thank you! :)
When a random RUclips channel explains it better than your University Professor....
Keep it up!
Wow, thanks!
BAM!
:)
It is unfair that I can't give this video another like.
:)
it is good to listen to your music in your website after watching this clear-explained video. thanks a lot.
Thank you so much! :)
I love this guy's shtick. Corny, slightly annoying music, although I'm sure he is a great musician. Slightly condescending voice when he goes over the material... like "I'm making this so fucking easy for you... you can't possibly not understand this". It's actually quite calming. He speaks slowly too. You don't have to constantly pause his videos. I understand everyone of his videos. If I don't, it's because I didn't yet watch any prerequisite videos that he tells you at the beginning to watch.
He never takes for granted that you understand some detail. This is the BIGGEST freakin' mistake of educators. Some damn variable in a formula that they forget to explain. Also, he will use the simplest example possible so that you understand.
I am returning to school, grad school in the ML track for computer science. I don't remember much of the math that I took 20 years ago. This guy is a lifesaver. Wish I watched these when I started. I will be watching all of his videos.
After I graduate and make some money, I'm sending him some bucks thru Patreon.
Thanks man!
BAM! Thank you very much! I think I must have "resting condescending voice" - because several people have made the comment that I sound a little condescending - but trust me this is not intentional! :)
@@statquest It's actually reassuring. You know, when you are talking to someone who is freaking out? And you make it sound like "Dood, this not that hard."
@@Steve-3P0 Nice! :)
Thank you josh and the FFGDUNCCH (the friendly folks from the genetics department at the university of north carolina at chapel hill)
Triple bam! :)
Just wow thanks Josh. You are just great. One doubt however, if k values are large will outliers not affect my algo? Effect of outliers in knn? Please answer.
I believe that large values for K will provide some protection from outliers.
When I search for something and find it on StatQuest channel. Super BAM!!
YES!
Your videos are K-nearest perfection :)
Ha! Very funny.
@@statquest Noice 👍 Thanks 👍
Bam!! Now ik things better than before.... Double Bam!! .
Hooray!
Your videos are sooo great, I can't stop watching 💖💖 thank you
Hooray!!!!
StatQuest with Josh Starmer can you add an ICA as well?
It's on the to-do list, but it might be a while before I get to it.
StatQuest with Josh Starmer 😔😕 that's sad, but i look forward to it. You explain beautifully sir! 💪🏼👊🏼
This channel is salt of the Earth
Thanks!
This is by far the best video on KNN algo ! Thanks Josh
You are doing awesome work Sir..have watched your other videos as well..very intuitive and logically explained
Thank you! This helped me so much in understanding KNN faster :D
Hooray!!! :)
These videos are just amazing and clearly are extremely successful in simplifying topics that are usually thought of as difficult. Can you please also make videos on its code in python/R..? and of naive bayes too maybe. That would be super useful. Thank you very much for this level of awesome content.
I'll keep that in mind.
Clear and concise explanation. Thank you :)
Thanks! :)
Thank you for your Clear explanation.
You're welcome! :)
Came here to learn kNN, ended up learning guitar!
bam!
Your video is amazing as always... It would be great if you can include how to choose the value for 'k' and evaluation metrics for kNN. Also, if I understand it right, there is no actual "training" happening in kNN. It is about arranging the points on the cartesian plane and when a new data point comes, it will again be placed on the same plane and depending on the value of "k", it will be classified. Correct me if I'm wrong.
Hi. Yes, you are right. KNN is easy to implement and understand and has been widely used in academia and industry for decades. You may utilise the cross-validation technique and the validation datasets to select the value for k.
Thank you so much. So useful honestly - i didnt get this from a 2 hour lecture
Glad it was helpful!
This channel is GOD SENT. Period.
Thanks!
awesome! You should do a quadratic discriminant analysis to go with your awesome one on LDA
Hello Josh, can you please explain why 3 nearest neighbours are Orange and 1 nearest neighbour is green (the red one looks closest to the black spot to me)?
I might have misunderstood the meaning of k nearest neighbours, though
PS: loved your explanation, thank you!
What time point, minutes and seconds, are you asking about?
@@statquest I'm sorry I forgot to mention it, it's at 2:29
@@sarvesh_7736 Of the 11 colored dots that are closest to the big black dot, 7 of them are red, 3 of them are orange and 1 one of them is green.
Intro "Na na na na na na.. StatQuest"
:)
Thank you so much for saving our time sir❤ love from Srilanka 🇱🇰
bam!
sad that finding this now. happy that I found it , rather not finding ever :)
better late than never! :)
I stopped watching the videos given by my college now when I stumbled upon these
BAM! :)
I can't believe how good you are at explaining this. wow!!!
bam!
thank you so much for this video! i have my midterm tomorrow and im so scared :(
Good luck!!
My 10 year old hums statquest song made me realise I my new obsession with this
bam!
Ohhh.. I got confused with K-means clustering and K-nearest neighbors......
Same doubt .
Is it actually based on just vote? or are the votes weighted based on the distance to the data point?
You can do it either way. If you add weights, then it is "weighted K-NN"
Man, you are a legend, if I pass from the exam on Monday (which I am pretty hopeless), I will buy one of your shirts next month
Hooray! Good luck with your exam! :)
@@statquest Hey, I failed :D but still, I learnt a lot, thanks!
@@eltajbabazade1189 Better luck next time! :)
@@eltajbabazade1189 I hope you graduated successfully 🙂.
Hello Josh how are you. I was wondering if you may kindly explain the Naive Bayes, to be clearly explained :)
Thank you, very clear and to the point explanation !
Good stuff, thanks! Do you have any videos about survival analysis?
Hail Joshua!!
BAM! :)
WOWW! This was super helpful!
Thanks Josh!
Glad it was helpful!
is considering this my favourite channel makes me a nerd ?
It makes you awesome! :)
Ohhh man this so simple
Thqqq for this type of explanation
Most welcome 😊
I liked the video immediately after hearing the guitar intro
bam! :)
When we have categorical variable like Yes/No or type of job (which can take four values: business, healthcare, engineering, or education), how can we calculate distances? Is knn useful at all?
If there is a distance metric, then it KNN will work, and there are distance metrics for categorical variables. See stackoverflow.com/questions/2981743/ways-to-calculate-similarity/2983763#2983763
awesome explanation ! thank you so much!
Thank you! :)
I am brushing up on my ML terminology and StatQuest always comes to the rescue!! BAM!
bam!
Where would I be without StatQuest? Luckily, I now have the statistical tool to estimate this!
bam!
Wow! such a great explainer
Glad you think so!
watch for the stats, stay for the intro songs
bam! :)
Quite dissapointed at the lake of bams
:)
How should i cite this information?
You can cite StatQuest the same way you would cite any other website or RUclips video. Here is an example:
StatQuest. “R-squared explained” RUclips, Joshua Starmer, 3 Feb. 2015, ruclips.net/video/2AQKmw14mHM/видео.html.
That opening banjo solo is prettt sweet.
Thanks!
lifesaver! thank you!
Glad it helped!
I love you sir! Your video save my life!
Happy to help!
You're a legend ! Thank you :)
Thanks!
BAM! Amazing explanation!
Thanks!
one video explained better than a whole semester
Awesome! :)
Thanks for the very informative info ! Though I have a question , if my dataset is filled with just categorical string data. So no numerical data . Is there a way I can still use knn to predict ? I heard about encoding the string to numerical value but that seems very complex with big dataset .
If you use R, then you can use a Random Forest to cluster anything and then apply KNN to that clustering: ruclips.net/video/sQ870aTKqiM/видео.html If you don't use R, you can use target encoding: ruclips.net/video/589nCGeWG1w/видео.html
Thanks for your youtube :)
No problem 😊
You are amazing! Thank u so much.
Cheers from BRAZIL
Muito obrigado! :)
finally...! i can pass my exams..!1
Bam! :)
Amazing explanation! Thank you!
Good job ! I loved the videooo :)
Thanks!
Hi . Thanks for video . About the concept of KNN , how the location of unknown cell change in scatter plot . You must change the location of that? And second question, we should change the value of k to reach best k ?
The location of the "unknown cell" is fixed - it does not change. Just the classification changes. I offer a few thoughts about how to pick 'k' at 4:12, but, other than that, you can use cross validation: ruclips.net/video/fSytzGwwBVw/видео.html
StatQuest with Josh Starmer I got it . Thank you 🙏 😀
I like your bandcamp!
Hooray! Thank you! :)
Loved it.... Thank you 😊
Glad you enjoyed it!
Explanation was very very very good. Easily understandable by anyone almost.
Can you please do a video on KFold and StratifiedKFold with an example using python. Also, can you explain cross_val_score in details
I have a video on cross validation that covers the concepts in K-Fold cross validation. That might not be exactly what you are looking for, but just in case, here is the link: ruclips.net/video/fSytzGwwBVw/видео.html
very machine learning voice 😂
:)
It was super simple indeed!
:)
Very well explained and loved your uke intro by the way :)
Thank you!
Very clear, I got the idea of this concept right away.
Well done, thanks!
THanks!
Great explanation! BAM! Great illustrations! Double BAM!!
Thank you very much! :)
Great video man
Thanks!
Omg thank you so much
No problem!
Simple and Clear explanation. Thank you!
Thanks!
I don't understand the purpose of using PCA. Since we have a dataset of known categories, why can't we directly calculate the Euclidean distance between samples of unknown categories and samples of known categories and determine the category to which they belong, like K-means. This sounds stupid but I'm actually a little confused.
PCA can help remove noise from the data and it can also make it easier to see what the data look like. That said, it's totally optional.
Please do a video on K-Medoid
I'll keep that in mind.
THANK YOU!
YOU HAVE SAVED ME :D
Awesome! :)
thank you so much.This was well explained.
Thanks!
Hey Josh! This is just a thank you note saying if I pass the upcoming exam, then it would be all because of you! ❤
Good luck!!! Let me know how it goes!
@@statquest It went well, thank you! Hopefully I get good grades. I was thinking of suggesting that it would be great if you could cover Markov Chain Monte Carlo and related topics. Thank you again! Your channel has been incredibly helpful!
@@suparnaroy2829 I'm glad it went well! And I'll keep those topics in mind.
00:10 K-nearest neighbors is a simple algorithm for classifying data.
00:50 Clustering data using PCA and classifying new cell type
01:29 K-nearest neighbors classifies new data based on nearest annotated cells.
02:12 K-nearest neighbors algorithm assigns a category based on the majority of nearest neighbors' votes.
02:59 K-nearest neighbors algorithm classifies unknown points based on nearest neighbors
03:40 K-nearest neighbors can avoid ties by using an odd K value.
04:22 Choosing the best value for K is crucial for K-nearest neighbors.
05:01 Categories with few samples are outvoted
You forgot the bam! :)
Summarised in a very short video....just perfect
Thank you! :)
Elegant! BAM
Hooray! Thanks! :)
Dang. Simple and to the point! Thank you!
Thanks!
Is it possible for you to add naive bayes classifier to the statquests?
It's on the to-do list, and I just bumped it a little closer to the top with your vote.
Thank you!!!
Low values of K(k=1 or K=2) can be noisy. But in your example, the cells are evenly space. K=1 seems to be perfect or do not have outliers. Or do you mean that in real cases, there is a cluster and not evenly spread like yours?
This is just an example of the principals behind k-nearest neighbors.
sir can we replace NaN value of column by mean in such a way that if other parameter value is in a particular range than find the mean and replace .
Example..if column BMI has NaN value then if age of that person is 45 then we first find the mean BMI of people with a age of range 40 to 50 and replace with this.Similarly,for other person have NaN BMI ... then first check the age of that person and set an interval age and find mean and replace...
There are a million ways to fill in missing data. Maybe one day I'll do a whole video on the topic.
watching at 2x speed is recommended. :)
2x BAM!
Sorry, but you lost me at 0:54. What is PCA? Do you think expanding that abbreviation will help new comers like me?
Here's a video that explains PCA: ruclips.net/video/FgakZw6K1QQ/видео.html
That is awsom how you explain this topics. One suggestion, you could show how the 7 nearest ist red, 3 nearest ist orange and 1 nearest is green for the point in the middle. By my eyes, the 1 nearest neigbour ist still red! and it makes me confuse what does nearest means actually :)
What time point, minutes and seconds, are you referring to?