If you want to see why Lasso can set parameters to 0 and Ridge can not, check out: ruclips.net/video/Xm2C_gTAl8c/видео.html Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
Good video, but didn't really explain how LASSO gets to make a variable zero. What's the difference between squaring a term and using the absolute value for that?
Intuitively, the closer slope gets to zero, the square of that number becomes insignificant compared to the increase in the sum of the squared error. In other words, the smaller you slope, the square gets asymptotically close to 0 because it can't outweigh the increase in the sum of squared error. In contrast, the absolute value adds a fixed amount to the regularization penalty and can overcome the increase in the sum of squared error.
@@theethatanuraksoontorn2517 Maybe this discussion on stack-exchange will clear things up for you: stats.stackexchange.com/questions/151954/sparsity-in-lasso-and-advantage-over-ridge-statistical-learning
@@statquest I didn't reply before, but the answer really helped me a lot, with basic machine learning and now artificial neural networks, thank you very much for the videos and the replies :D
I am so happy to easily understand these methods after only a few minutes (after spending so many hours studying without really understanding what it was about). Thank you so much, your videos are increadibly helpful! 💯☺
Hi man, really LOVE your videos. Right now I'm studying Data Science and Machine Learning and more often than not your videos are the light at the end of the tunnel, sot thanks!
Came here because I didn't understand it at all when my professor lectured about LASSO in my university course... I have a much better understanding now thank you so much!
Yeahhhh!!! I was the first to express Gratitude to Josh for this awesome video!! Thanks Josh for posting this and man! your channel is growing.. last time, 4 months ago it was 12k. You have the better stats ;)
NOBODY IS GOING TO TALK ABOUT THE EUROPEAN / AFRICAN SWALLOW REFERENCE ????are you all dummies or something ? It made my day. Plus, video on top, congratulation. BAMM !
Me and my friend are studying. When the first BAM came, we fell for laught for about 5min. Then the DOUBLE BAM would cause a catrastofic laughter if we didn't stop it . I want you to be my professor please!
Thank you, Josh, for this exciting and educational video! It was really insightful to learn both the superficial difference (i.e. how the coefficients of the predictors are penalized) and the significant difference in terms of application (i.e. some useless predictors may be excluded through Lasso regression)!
Thank you so much for the video ! I have watched several your videos and I prefer to watch your video first then see the real math formula. When I did that, the formula became so easier and understandable! For instance, I don't even know what does 'norm' is, but after watching your video then it would be very easy to understand!
Hi Josh! I am a big fan of your videos and it is clearly the best way to learn machine learning. I would like to ask you if you will be uploading videos relating to deep learning and NLP as well. If so, that will be awesome. BAM!!!
Right now I'm finishing up Support Vector Machines (one more video), then I'll do a series of videos on XGBoost and after that I'll do neural networks and deep learning.
Hi. Your videos are so helpful. I really appreciate you spend time doing them. I have one question related to this video: Is the result of Lasso Regression sensitive to the unit of variables? For example in the model: size of mice = B0 + B1*weight + B2*High Fat Diet + B3*Sign + B4*AirSpeed + epsilon Suppose the original unit of weight in the data is gram. If we divide the weight by 1,000 to get unit in kilogram, is the Lasso Regression different? As I understand, the least square estimated B1-kilogram should be 1,000 times higher than the B1-gram. Therefore, B1-kilogram is more likely to be vanished in Lasso, isn't?
I am also looking for the answer to this. I'm just using my intuition here, but here's what I think. The least important variables have terrible predictive value so the residuals along these dimensions are the highest. If we create a penalty for introducing these variables (especially with a large lambda that outweighs/similar in magnitude to the size of these residuals squared), decrease in coefficient of these "bad predictors" will cause comparatively smaller increase in residuals compared to the decrease in penalty due to the randomness of these predictors. In contrast, the penalty for "good predictors" (which are less random) will cause significant change in residuals as we decrease its coefficients. This would probably mean that these coefficients would have to undergo smaller change to account for the larger increase in residuals. This is why the minimisation will reduce the coefficients of "bad predictors" faster than "good predictors. I take this case would be especially true when cross-validating.
if you draw the curves of y=x and y=x^2, you will find the gradient will vanish for y=x^2 near origin point, hence very hard to be decreased to zero if using optimizing approach like SGD.
Great work, Thank you Josh, I'm trying to connect ideas from different perspectives/angles, Does the lambda here somehow related to Lagrange multiplier ?
Thanks a lot! Amazing explanation! Please, continue the great work and add more on statistics, probability in general and machine learning in particular. Sinse Data Science suppose to have a great future, I am certain that your channel also will prosper a great deal!
love the work, i remember reading books about linear regresion, when they spent like 5 pages for these 2 topics but i still have no clue what they really do =))
I am sorry but I'm not able to figure out why (regardless of the approach - ridge and lasso), the 'good' parameters 'slope' and 'diet difference' will behave differently than the other two silly ones. I don't understand this since you are applying the same 'lambda' and absolute value for all 4 parameters. It'd be really kind of you to clear my silly doubt. Thanks!
this is awesome thank you so much for this u explained it so well . I will recommend this video to every one I know who is interested . I also watched your lasso video and it was just as good thank you
I love your style of explaining! You leave enough time for anyone to take in all information while talking. Sometimes it feels like you are trying to teach little kids, but it actually just works. I often watch other teaching videos and can't remember most of it afterwards, but I can remember almost everything that you are saying after the first time watching. Amazing job! I have one question though. You were saying that the regression model in the beginning had low bias and high variance. Does it not have high bias? As far as I know bias represents the expected generalization (or test) error, if we were to fit a very large training set. If we fit that simple model to a lot of data, the generalization error would be rather high, because it could not capture the true patterns in the data.
I'm glad you like the videos! In ML, there are specific meanings for bias and variance that are a little bit different from what you are using and I explain in this StatQuest: ruclips.net/video/EuBBz3bI-aA/видео.html
love your videos.... extremely helpful and cristal clear explained.... but your songs..... let's say you have a very promising career as a statistician... no question
ThankYou Josh for another amazing Statquest! I want to know if L1 and L2 regularization terms should be used together or separately What should be the standard approach. I'm specifically using boosting algorithms.
Both the Ridge and Lasso videos made me want to cry. (Know you aren't alone if anyone else feels the same.) Also noteworthy: Ridge Regression avoids problems introduced by having more observations than predictor variables, or when multicollinearity is an issue. This example avoids either condition. Triple Bam. (Obviously, I am taking the definition too literally. It's a relative statement, re: the vars to observations ratio. )...Nevertheless, there's no end to my confusion. I was approaching "understanding", using the ISLR book....but you can actually get two different perspectives on the same topic, and then be worse off due to variance in how the concepts are presented. That said, you're still awesome, StatsQuest, and you are invited to play guitar at my funeral when I end things from trying to learn ML.
Hi thanks I think the videos are great and also I like your songs. You are very talented. In looking at this video I was thinking that Lasso regression can be used as a form of variable selection. Is this a good idea? So basically at first you include all the predictors then the lasso will tell which variable you need to get rid off. Does this make sense?
Thanks very much for this video, it really helps me with the concept of the Ridge Regression and the Lasso Regression. I have a silly question: why the parameter in the Ridge Regression cannot shrink to zero but in Lasso, they can?
That's not a silly question at all, and there are lots of websites that dive into that answer. I'd just do a google search and you should find what you're looking for.
I have seen some articles mentioning that Ridge Regression is better in handling multicollinearity between variables as compared to Lasso. But i am not sure of the reason why. Since the difference between Lasso and Ridge is just the way it penalized the coefficients.
Awesome video! Follow up question - in your examples, the initial models all overestimated the impact of weight on size, and ridge/lasso regression corrected it by picking a new model with lower coefficients for weight. What happens if the initial model under-estimates the coefficient for weight? Would ridge/lasso introduce more variance by lowering the coefficients further?
Lasso and Ridge Regression both use cross validation, so that means that they use a wide variety of "training" data to make the estimates - meaning that they will sometimes underestimate and sometimes overestimate. The Lasso and Ridge regression help them find the happy medium.
@@yellowburros123 Depending on the model, Ridge and Lasso can increase the coefficients for some variables. This is illustrated in the Introduction to Statistical Learning in R www-bcf.usc.edu/~gareth/ISL/ISLR%20Seventh%20Printing.pdf on page 216.
at the end of the last video [regularization part 1: ridge (l2) regression], you mentioned to solve the problem that how to estimate 10000 parameters with 500 samples and will talk about it in the next one, and after finishing this video I was still wondering how to deal with it...🤣🤣 am i looking at these videos in the wrong order or what?
You have the correct order. Unfortunately, all I have had time to do is provide a general intuition on how cross validation is used to find an optimal line, even when we don't have enough data.
Hi Josh, could you please make video(s) on Correspondence Analysis, Chi-Square distance and Multiple contingency table? Learning from your videos are much efficient than several books combined :) !
If you want to see why Lasso can set parameters to 0 and Ridge can not, check out: ruclips.net/video/Xm2C_gTAl8c/видео.html
Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
Love how you keep these videos introductory and don't go into the heavy math right away to confuse;
Love the series!
Thank you!
One of the best explanation of Ridge and Lasso regression I have seen till date... Keep up the good work....Kudos !!!
Thanks! :)
I am eternally grateful to you and those videos!! Really saves me time in preparing for exams!!
Happy to help!
My teacher is 75 years old, explained us Lasso during one hour , without explaining it. But this is a war I can win :), thanks to your efforts.
I love it!!! Glad my video is helpful! :) p.s. I got the joke too. Nice! ;)
Why is this scenario many times the reality? Also, I check StatQuest's vids very often to really understand the things. Thanks @StatQuest
(Possible) Fact: 78% of people who understand statistics and machine learning attribute their comprehension to StatQuest.
bam! :)
Hi, I can't thank you enough for explaining the core concepts in such short amount of time. Your videos help a lot! My appreciations are beyond words.
Thank you!
Don't think your Monty Python reference went unnoticed
(Terrific and very helpful video, as always)
Thanks so much!!! :)
Oh it absolutely did. And it was much loved!
That "Bam???" cracks me up. Thanks for your work!
:)
Good video, but didn't really explain how LASSO gets to make a variable zero. What's the difference between squaring a term and using the absolute value for that?
Intuitively, the closer slope gets to zero, the square of that number becomes insignificant compared to the increase in the sum of the squared error. In other words, the smaller you slope, the square gets asymptotically close to 0 because it can't outweigh the increase in the sum of squared error. In contrast, the absolute value adds a fixed amount to the regularization penalty and can overcome the increase in the sum of squared error.
@@theethatanuraksoontorn2517 Maybe this discussion on stack-exchange will clear things up for you: stats.stackexchange.com/questions/151954/sparsity-in-lasso-and-advantage-over-ridge-statistical-learning
@@statquest Thanks for reading the comments and responding!
@@programminginterviewprep1808 I'm glad to help. :)
@@statquest I didn't reply before, but the answer really helped me a lot, with basic machine learning and now artificial neural networks, thank you very much for the videos and the replies :D
Some video ideas to better explain the following topics:
1. Monte Carlo experiments
2. Bootstrapping
3. Kernel functions in ML
4. Why ML is black box
OK. I'll add those to the to-do list. The more people that ask for them, the more I'll priority they will get.
@@statquest That is great! keep up the great work!
@@statquest yes we need it please do plsssssssssssssssssssssssssssssssss
plsssssssssssssssssssssssssssssssssssssssssssssss
Bootstrapping is explained well in Random Forest video.
Do it for us... thanks good stuff
I am so happy to easily understand these methods after only a few minutes (after spending so many hours studying without really understanding what it was about). Thank you so much, your videos are increadibly helpful! 💯☺
Great to hear!
This channel is pure gold. This would have saved me hours of internet search... Keep up the good work!
Thank you! :)
Hi man, really LOVE your videos. Right now I'm studying Data Science and Machine Learning and more often than not your videos are the light at the end of the tunnel, sot thanks!
The difference between BAM??? and BAM!!! is hilarious!!
:)
@@statquestCan you please explain how the irrelevant parameters "shrink"? How does Lasso go to zero when Ridge doesn't?
@@SaiSrikarDabbukottu I show how it all works in this video: ruclips.net/video/Xm2C_gTAl8c/видео.html
Came here because I didn't understand it at all when my professor lectured about LASSO in my university course... I have a much better understanding now thank you so much!
Awesome!! I'm glad the video was helpful. :)
Josh - as always your videos are brilliant in their simplicity! Please keep up your good work!
Thanks, will do!
Every time I think your video subject is going to be daunting, I find you explanation dispel that thought pretty quickly. Nice job!
Yeahhhh!!! I was the first to express Gratitude to Josh for this awesome video!! Thanks Josh for posting this and man! your channel is growing.. last time, 4 months ago it was 12k. You have the better stats ;)
Hooray! Yes, the channel is growing and that is very exciting. It makes me want to work harder to make more videos as quickly as I can. :)
@@statquest please keep on going... You are our saviour
I am eternally grateful to you. You've helped immensely with my last assessment in uni to finish my bachelors
Congratulations!!! I'm glad my videos were helpful! BAM! :)
Your intro songs reminds me of Pheobe from the TV show "Friends", and the songs are amazing for starting the videos on a good note, cheers!
You should really check out the intro song for this StatQuest: ruclips.net/video/D0efHEJsfHo/видео.html
NOBODY IS GOING TO TALK ABOUT THE EUROPEAN / AFRICAN SWALLOW REFERENCE ????are you all dummies or something ? It made my day. Plus, video on top, congratulation. BAMM !
bam!
Airspeed of swallow lol. These videos are really helping me a ton, very simply explained and entertaining as well!
Glad you like them!
I really appreciated the inclusion of swallow airspeed as a variable above and beyond the clear-cut explanation. Thanks Josh. ;-)
:)
Me too!
The beginning songs are always amazing hahaha!!
Awesome! :)
So easy to understand. And I like the double BAM!!!
Thanks!
This is brilliant. Thanks for making it publicly available
You're welcome! :)
Statquest is like Marshall Eriksen from HIMYM teaching us stats. BAM? Awesome work Josh.
Thanks!
Just love the way you say 'BAM?'.....a feeling of hope mixed with optimism, anxiety and doubt 😅
:)
Me and my friend are studying. When the first BAM came, we fell for laught for about 5min. Then the DOUBLE BAM would cause a catrastofic laughter if we didn't stop it . I want you to be my professor please!
BAM! :)
Thank you, Josh, for this exciting and educational video! It was really insightful to learn both the superficial difference (i.e. how the coefficients of the predictors are penalized) and the significant difference in terms of application (i.e. some useless predictors may be excluded through Lasso regression)!
Double BAM! :)
Thank you so much for the video !
I have watched several your videos and I prefer to watch your video first then see the real math formula. When I did that, the formula became so easier and understandable!
For instance, I don't even know what does 'norm' is, but after watching your video then it would be very easy to understand!
Awesome! I'm glad the videos are helpful. :)
Excelent video Josh! Amazing way to explain Statistics Thank you so much! Regards from Querétaro, México
Muchas gracias! :)
Great people know subtle differences which is not visible to common eyes
love you sir
Thanks!
Bam! I appreciate the pace of the videos. Thanks for doing this.
Thanks! :)
Explained in a very simple yet very effective way! Thank you for your contribution Sir
Hooray! I'm glad you like my video. :)
I came for the quality content, fell in love with the songs and bam.
BAM! :)
Thanks for posting, my new favourite youtube channel absolutely !!!!
Wow, thanks!
Thx very much. Clear explanation for these similar models. Great video I will conserve forever
why can't ridge reduce weight/parameter to 0 like lasso?
a man of his word...very clearly explained!
Thank you! :)
Thank you so much for making these videos! Had to hold a presentation about LASSO in university.
I hope the presentation went well! :)
@@statquest Thx. It did :)
The other day, I had homework to write about Lasso and I struggled.. wish I had seen this video a few days earlier.. Thank you as always!
Great video, clear explanation, loved the Swallows reference! Keep it up! :)
Awesome, thank you!
A StatQuest a day, keeps Stat fear away!
I love it! :)
Your videos make it so easy to understand. Thank you!
Thank you! :)
Harvard should hire you. Your videos never fail me!
Thank you for such great content!
Thank you very much!!!
Hi Josh! I am a big fan of your videos and it is clearly the best way to learn machine learning. I would like to ask you if you will be uploading videos relating to deep learning and NLP as well. If so, that will be awesome. BAM!!!
Right now I'm finishing up Support Vector Machines (one more video), then I'll do a series of videos on XGBoost and after that I'll do neural networks and deep learning.
StatQuest with Josh Starmer Thanks Josh for the updates. I’ll send you request at Linkedin.
One more use case of Ridge/Lasso regression is 1) When data points are less 2) High Multicollinearity between variables
You have a gift for teaching! Excellent videos!
Thanks! :)
I prefer the intro where is firmly claimed that StatQuest is bad to the bone. And yes I think this is fundamental.
That’s one of my favorite intros too! :)
But I think my all time favorite is the one for LDA.
Yes I agree! Together these two could be the StatQuest manifesto summarising what people think about stats!
So true!
That Monty Python reference though... good video btw :)
Ha! I'm glad you like the video. ;)
Hi Josh, Thanks for clear explanation on regularization techniques. very exciting. God bless for efforts.
Glad you enjoyed it!
Incredible great explanations of regularization methods, thanks a lot.
Thanks! :)
Hi. Your videos are so helpful. I really appreciate you spend time doing them.
I have one question related to this video: Is the result of Lasso Regression sensitive to the unit of variables?
For example in the model: size of mice = B0 + B1*weight + B2*High Fat Diet + B3*Sign + B4*AirSpeed + epsilon
Suppose the original unit of weight in the data is gram. If we divide the weight by 1,000 to get unit in kilogram, is the Lasso Regression different?
As I understand, the least square estimated B1-kilogram should be 1,000 times higher than the B1-gram. Therefore, B1-kilogram is more likely to be vanished in Lasso, isn't?
Million BAM for this channel 🎉🎉🎉
Thank you!
Seriously the best videos ever!!
Thanks!
How do Ridge or Lasso know which variables are useless? Will they not also shrink the parameter of important variables ?
I am also looking for the answer to this. I'm just using my intuition here, but here's what I think. The least important variables have terrible predictive value so the residuals along these dimensions are the highest. If we create a penalty for introducing these variables (especially with a large lambda that outweighs/similar in magnitude to the size of these residuals squared), decrease in coefficient of these "bad predictors" will cause comparatively smaller increase in residuals compared to the decrease in penalty due to the randomness of these predictors. In contrast, the penalty for "good predictors" (which are less random) will cause significant change in residuals as we decrease its coefficients. This would probably mean that these coefficients would have to undergo smaller change to account for the larger increase in residuals. This is why the minimisation will reduce the coefficients of "bad predictors" faster than "good predictors. I take this case would be especially true when cross-validating.
if you draw the curves of y=x and y=x^2, you will find the gradient will vanish for y=x^2 near origin point, hence very hard to be decreased to zero if using optimizing approach like SGD.
Dude you are an absolute lifesaver! keep it up!!!
Hooray! I'm glad I could help. :)
Hahaha.. That moment you said BAM??? I laughed out loud 🤣🤣🤣
:)
Great work, Thank you Josh,
I'm trying to connect ideas from different perspectives/angles, Does the lambda here somehow related to Lagrange multiplier ?
I'm not sure.
Thanks a lot! Amazing explanation! Please, continue the great work and add more on statistics, probability in general and machine learning in particular. Sinse Data Science suppose to have a great future, I am certain that your channel also will prosper a great deal!
Thank you! :)
Amazing video, explanation is fantastic. I like the song along with the concept :)
Bam! :)
Best youtube channel
Thank you! :)
love the work, i remember reading books about linear regresion, when they spent like 5 pages for these 2 topics but i still have no clue what they really do =))
Glad it was helpful!
Love the fact that you reply to every single comment here in YT haha
me: wathcing these videos in full panic
video: plays calming music
me: :)
bam! Good luck! :)
I am sorry but I'm not able to figure out why (regardless of the approach - ridge and lasso), the 'good' parameters 'slope' and 'diet difference' will behave differently than the other two silly ones. I don't understand this since you are applying the same 'lambda' and absolute value for all 4 parameters. It'd be really kind of you to clear my silly doubt. Thanks!
Thanks for the Video. They make difficult concepts seem really easy..
Thank you! :)
@@statquest Can u make a similar video for LSTM?
this is awesome thank you so much for this u explained it so well . I will recommend this video to every one I know who is interested . I also watched your lasso video and it was just as good thank you
Thank you very much! :)
Amazing explanation. Loved the Monty Python reference :D
:)
Thank you once again Josh!
bam!
I love your style of explaining! You leave enough time for anyone to take in all information while talking. Sometimes it feels like you are trying to teach little kids, but it actually just works. I often watch other teaching videos and can't remember most of it afterwards, but I can remember almost everything that you are saying after the first time watching. Amazing job!
I have one question though. You were saying that the regression model in the beginning had low bias and high variance. Does it not have high bias? As far as I know bias represents the expected generalization (or test) error, if we were to fit a very large training set. If we fit that simple model to a lot of data, the generalization error would be rather high, because it could not capture the true patterns in the data.
I'm glad you like the videos! In ML, there are specific meanings for bias and variance that are a little bit different from what you are using and I explain in this StatQuest: ruclips.net/video/EuBBz3bI-aA/видео.html
Wow! so easy to understand this! Thanks very much!
Thanks!
Hooray!!!! excellent video as always
Thank you!
Hooray, indeed!!!! Glad you like this one! :)
Great....please continue to learn other models...thank you so much.
Thanks!
Brilliant explanation
didnt need to check out any other video
Thank you!
so incredible, so well explained
Thanks!
Thank you so much for these videos you are a literal godsend. You should do a video on weighted least squares!!
I enjoy the content and your jam so much! '~Stat Quest~~'
Thanks!
My favourite youtuber!
Thank you! :)
love your videos.... extremely helpful and cristal clear explained.... but your songs..... let's say you have a very promising career as a statistician... no question
;)
Finally, I found 'The One'!
:)
ThankYou Josh for another amazing Statquest! I want to know if L1 and L2 regularization terms should be used together or separately What should be the standard approach. I'm specifically using boosting algorithms.
Usually they are used together. See: ruclips.net/video/1dKRdX9bfIo/видео.html
@@statquest thankyou
Good question! that combination is called Elastic Net regularization.
Both the Ridge and Lasso videos made me want to cry. (Know you aren't alone if anyone else feels the same.) Also noteworthy: Ridge Regression avoids problems introduced by having more observations than predictor variables, or when multicollinearity is an issue. This example avoids either condition. Triple Bam. (Obviously, I am taking the definition too literally. It's a relative statement, re: the vars to observations ratio. )...Nevertheless, there's no end to my confusion. I was approaching "understanding", using the ISLR book....but you can actually get two different perspectives on the same topic, and then be worse off due to variance in how the concepts are presented. That said, you're still awesome, StatsQuest, and you are invited to play guitar at my funeral when I end things from trying to learn ML.
(gonna check the StatsExchange link down below that you provided. Thank you, sir!!!)
Hi thanks I think the videos are great and also I like your songs. You are very talented.
In looking at this video I was thinking that Lasso regression can be used as a form of variable selection. Is this a good idea? So basically at first you include all the predictors then the lasso will tell which variable you need to get rid off. Does this make sense?
Thanks very much for this video, it really helps me with the concept of the Ridge Regression and the Lasso Regression. I have a silly question: why the parameter in the Ridge Regression cannot shrink to zero but in Lasso, they can?
That's not a silly question at all, and there are lots of websites that dive into that answer. I'd just do a google search and you should find what you're looking for.
this also make me muddle
I have seen some articles mentioning that Ridge Regression is better in handling multicollinearity between variables as compared to Lasso. But i am not sure of the reason why. Since the difference between Lasso and Ridge is just the way it penalized the coefficients.
Fantastic videos - very well explained!
Thank you! :)
wonderfully explained
Thank you! :)
Great video! The topic is really well explained
Thank you!
Thankyou Sir ! Great Help.
Thanks! I finally understand how they shrink parameters!
The best explanation ever.
Thank you Josh for sharing this video. Could you please do a video on Bayesian statistics and Monte Carlo methods?
I hope to do bayesian statistics soon.
Is there a more detailed explanation as to how some feature weights become zero in the case of Lasso, and why that cannot happen in Ridge? Thanks.
Yes, see: ruclips.net/video/Xm2C_gTAl8c/видео.html
@@statquest Thanks! I watched the video, but I am still not sure why there is a kink in the case of Lasso. What exactly creates that kink?
@@rakeshk6799 The absolute value function.
Ah! A triple THANKSSSS!!!!. I finally got what they are really doing.
Hooray! :)
You are the best! I understand it now!
Thanks!
Awesome video! Follow up question - in your examples, the initial models all overestimated the impact of weight on size, and ridge/lasso regression corrected it by picking a new model with lower coefficients for weight. What happens if the initial model under-estimates the coefficient for weight? Would ridge/lasso introduce more variance by lowering the coefficients further?
Lasso and Ridge Regression both use cross validation, so that means that they use a wide variety of "training" data to make the estimates - meaning that they will sometimes underestimate and sometimes overestimate. The Lasso and Ridge regression help them find the happy medium.
@@statquest Got it! So it seems that Lasso and Ridge regressions can actually increase the coefficient values sometimes?
@@yellowburros123 Depending on the model, Ridge and Lasso can increase the coefficients for some variables. This is illustrated in the Introduction to Statistical Learning in R www-bcf.usc.edu/~gareth/ISL/ISLR%20Seventh%20Printing.pdf on page 216.
@@statquest , really appreciate it!
at the end of the last video [regularization part 1: ridge (l2) regression], you mentioned to solve the problem that how to estimate 10000 parameters with 500 samples and will talk about it in the next one, and after finishing this video I was still wondering how to deal with it...🤣🤣 am i looking at these videos in the wrong order or what?
You have the correct order. Unfortunately, all I have had time to do is provide a general intuition on how cross validation is used to find an optimal line, even when we don't have enough data.
Hi Josh, could you please make video(s) on Correspondence Analysis, Chi-Square distance and Multiple contingency table?
Learning from your videos are much efficient than several books combined :) !
I'm working on those, but the bad news is that they will not be ready for a long time.... :(