Double, wobble bam! With some ham cooked with eggs and spam on a frying pan sprayed with Pam! Wham! Now we have green eggs, spam, and ham! Enjoy! Mr. Sam I am.
Thank you so much for these videos! I am currently in a Machine Learning class and my teacher does not explain it very well. And I love your optimism. thank you again!
Hi, Josh! I'm an Economics student from Brazil. Even though it's only an undergraduate course (right after high school), it can be very demanding when it comes to Statistics, Econometrics and Machine Learning. Although I'm very new to this world, I can understand it pretty well with your help. Awesome videos!
that's the reason why I became a Patreon supporter, you have saved my life again!!!! And the cherry was the R functions explanations, I love this channel bro!
Most common methods used for local regression are 1- LOESS (locally estimated scatterplot smoothing) 2- LOWESS (locally weighted scatterplot smoothing)
Note that I think what you show is Robust LOESS (which also accounts for outliers, and gives the extra weight function for distance between y's and predicted y's). I think the LOWESS function in R does it by default, but the LOESS function (it's successor) doesn't do it by default. You have to set family="symmetric" in the LOESS to get it. Specifically these two models do the same: fit = loess(y~x, span=0.25, degree=1) fit2 = lowess(x, y, f=0.25, iter=0) And so does these: fit = loess(y~x, span=0.25, degree=1, family="symmetric") fit2 = lowess(x, y, f=0.25)
This was very helpful, but how do I get the actual regression function? Say I wanted to compute the approximated value for a x which is not a data point? In linear regression, you would get two parameters which define a line and use that line to compute the estimated y=f(x) for an x which is not a data point
I believe you just just connect each point that lowess creates with a straight line. So, for example, I have two consecutive x-axis values, x1, and x2, and corresponding lowess values y1, and y2, I just draw the line between (x1, y1) and (x2, y2) and if I want to make a prediction between x1, and x2, I plug the x value in to that equation for a line.
@@statquest Thanks for your reply, I did some further reading and a possibilty is to just use the method for an x which is not a data point. I.e. given a x which is not a data point select the region of data points z_i you want to include into the regression and calculate the weights based off the distance between the x and the z_i. From this you can calculate the regressionweights (beta_0, beta_1) just like you would with a data point z_i. For further reference see www.itl.nist.gov/div898/handbook/pmd/section1/dep/dep144.htm
@@Geologist997 All I have is this code that demonstrates how to use those functions in R: github.com/StatQuest/lowess_loess_demo/blob/master/lowess_loess_demo.R
Hello Josh, thanks for this video, it's awesome. I was just wondering if you had time to finally make a video about splines? Otherwise can you give me links of any documents that can help. I need especialy the part where we plot K+m graphs (K for knots and m for the order). Thanks in advance :)
How are we choosing the new points here? I am talking about all the "Red Xs". Will we just consider a point on the regression line that is perpendicular to the focal point?
The Red Xs are chosen after fitting a line to the data using weighted least squares. If you would like to learn about least squares, check out this video: ruclips.net/video/PaFPbb66DxQ/видео.html
hmmm this is interesting... i thot curves are fit using the kernel trick every time. this is new information. are you aware of any python library that can perform the same trick. and how does this compare to kernel tricks?
I just did a quick google search and it seems that some people use the statsmodels package to do Lowess curves in python: www.statsmodels.org/dev/generated/statsmodels.nonparametric.smoothers_lowess.lowess.html
Thanks so much for this!! Will we have more videos on curvilinear fitting? Do p values and inference work the same with curves ? In what way fiiting a curve is better than fitting a line?
Great video! Recently I've been working with the "Moving Least Square" concept which is widely used in computer graphics, especially image deformation. This video helps me understand this concept in statistical field. Thank you so much.
Could you please upload some videos clearly explaining Non linear methods such as polynomial regression,step function ,splines,Basis function & GAM . Thanks for making such awesome videos.👌
To quote the Wikipedia article on Polynomial Regression ( en.wikipedia.org/wiki/Polynomial_regression ): "Conveniently, [polynomial regression] models are all linear from the point of view of estimation, since the regression function is linear in terms of the unknown parameters β0, β1, .... Therefore, for least squares analysis, the computational and inferential problems of polynomial regression can be completely addressed using the techniques of multiple regression. This is done by treating x, x2, ... as being distinct independent variables in a multiple regression model." That means you can watch my videos on Linear Regression to learn about Polynomial Regression: ruclips.net/p/PLblh5JKOoLUIzaEkCLIUxQFjPIlapw8nU
Nice video. I now realise i made something similar with centre moving averages and different (linear/quadratic) best fit per point. As ever i suspected that a sign of a good idea, is that someone else has done it 10x better already.
This is why I love R. It has all sorts of fun functions like this. For anyone who is playing with timeseries data, eg sensor data, lowess/loess are smoothing functions and are great for preprocessing and cleaning data. To find out more checkout: en.wikipedia.org/wiki/Smoothing
Hello, thats so nice of you to teach us the Lowess fitting, I have a specific parameter which needs to be fitted. Yet i need to change the weighing relationship, can you help with some explanation or some material to study for it. Thankyou.
You could use them to make predictions with future data. That's essentially what a neural network does (however, it uses a different method to fit lines to the data).
Hi Josh, thanks for the video. Does LOWESS allow us to use set of external values to help with smoothing? Let's say I want to smooth some time-series data for one individual in a population. I'd like to do this while keeping in mind the overall population - so, when I smooth this individual, I'm doing it "along with" the population's mean time-series. Is something like this possible with LOWESS? Could I use the population's mean as a some kind of weight?
I am becoming a big StatQuest fan! Congrat for this great video. Just one question... According to what I understand the last points can be misleading because as we get more data, they will change. First we calculate last value of the curve, let's say corresponding to last point X(10), from points on the left, so X(5) to X(10). But as we get new points, let's say X(11), X(12), the value of the curve corresponding to X(10) will change because we now use points X(8) to X(12) in calculation. Am I correct? I hope this is clear... Thanks
Hey Josh great video! Any idea why loess (locally estimated scatterplot smoothing) and lowess (locally weighted scatterplot smoothing), are called a local regression ( en.wikipedia.org/wiki/Local_regression ) instead of a local autoregression?
@@sachinrana7554 Presumably you are asking for the formula that takes into account the weights, since I've already provided you with the intuition for what that formula does. Here's the formula: en.wikipedia.org/wiki/Local_regression
Hey Josh, Is Lowess and Loess the same thing as splines? If not and if you are taking suggestions, I think some content on interpolation or cubic and polynomial splines would be great!
It can be useful when your data has some unknown shape and you want to find the "top" or "bottom" of the shape, rather than just the maximum or minimum values.
I have a probably way too broad question here. I am reading ESL these days. And local linear regression and cubic smoothing spline are throwing around everywhere in the book. But I can't seem to have a good grasp of their relations and differences. What about them vs. neural network? Essentially, all of them are function approximators. Thank you very much!
Yes, they are all used for approximating non-linear functions. In theory Neural Nets are more flexible and can easily approximate more complicated non-linear surfaces. But I don't know all the details.
@@statquest thanks. is there anything that attempts to produce similar results, but only uses data points from the past at each step of the calculation?
Thank you for the great explanation of the LOWESS! Please explain the last iteration, after I have found the first new points you say that I have new weights, can you please explain in depth how do I use them for the next set of points?
Thanks a lot for you deep explanation. I am learning Data Science and wonder if you are familiar with good statistics courses that will also show me why Normal Distribution has such formula, kernels and other topics like CI's and all I need for ML. Please advise.
Would you be so kind to add boosting algorithms like xgboost? Your explanations are super simple and I would like to get an explanation from you how to use these algorithms.
@@statquest That’s how my marketing statistics instructor used to pronounce it, and that’s how we still pronounce it at the agency where I’m currently working as an analyst! 😁
Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
Thank you so much for your intuitive explanation! I'm honestly impressed by how well you explained it. Love the intro jingle as well :)
Glad you enjoyed it!
HI! Great video. I want to fit a confidence interval band to my loess curve. Any advice on how I do this?
Thank you! Easier than I thought :)
Hooray!!! That's great. :)
What is the source of the data for the noisy curve at 8:10 ? Thanks.
x
great
Thanks!
man you somehow make statistics feel intuitive somehow O_o
Cuz stat quest is cool
Bam!!!!
It's the intro
Double, wobble bam! With some ham cooked with eggs and spam on a frying pan sprayed with Pam! Wham! Now we have green eggs, spam, and ham! Enjoy! Mr. Sam I am.
Thank you so much for these videos! I am currently in a Machine Learning class and my teacher does not explain it very well. And I love your optimism. thank you again!
Glad I could help!
This method has helped me make an amazing stock prediction algorithm! I'm a millionaire now!! THANKS StatQuest!!!
Hooray!!! :)
million Bam!!!
wow, how did you do it? any advice on how to get started?
Hi, Josh! I'm an Economics student from Brazil. Even though it's only an undergraduate course (right after high school), it can be very demanding when it comes to Statistics, Econometrics and Machine Learning. Although I'm very new to this world, I can understand it pretty well with your help. Awesome videos!
Thank you very much and good luck with your courses! :)
I first like the video, then watch it! That's the quality of StatQuest.
BAM!
Thanks for the clear explanation. Just wonder in the 2nd iteration, how are the two different sets of weights used together. Are they multiplied?
To be honest, I don't know the answer to that question off the top of my head.
that's the reason why I became a Patreon supporter, you have saved my life again!!!! And the cherry was the R functions explanations, I love this channel bro!
Thank you very much!!!! I really appreciate your support!
Most common methods used for local regression are
1- LOESS (locally estimated scatterplot smoothing)
2- LOWESS (locally weighted scatterplot smoothing)
My understanding is that LOESS is a multivariable generalization of LOWESS.
5:34 Did you forget to move the window? I am confused by how you choose your points and windows.
I may have forgotten to move the window. Sorry for the confusion.
Dude! This is exactly what I need!
Sweeeeet!!!!
Your intro made my day on this pandemic season. Thanks!
Hooray! :)
Note that I think what you show is Robust LOESS (which also accounts for outliers, and gives the extra weight function for distance between y's and predicted y's). I think the LOWESS function in R does it by default, but the LOESS function (it's successor) doesn't do it by default. You have to set family="symmetric" in the LOESS to get it. Specifically these two models do the same:
fit = loess(y~x, span=0.25, degree=1)
fit2 = lowess(x, y, f=0.25, iter=0)
And so does these:
fit = loess(y~x, span=0.25, degree=1, family="symmetric")
fit2 = lowess(x, y, f=0.25)
Noted
This was very helpful, but how do I get the actual regression function? Say I wanted to compute the approximated value for a x which is not a data point?
In linear regression, you would get two parameters which define a line and use that line to compute the estimated y=f(x) for an x which is not a data point
I believe you just just connect each point that lowess creates with a straight line. So, for example, I have two consecutive x-axis values, x1, and x2, and corresponding lowess values y1, and y2, I just draw the line between (x1, y1) and (x2, y2) and if I want to make a prediction between x1, and x2, I plug the x value in to that equation for a line.
@@statquest Thanks for your reply, I did some further reading and a possibilty is to just use the method for an x which is not a data point. I.e. given a x which is not a data point select the region of data points z_i you want to include into the regression and calculate the weights based off the distance between the x and the z_i. From this you can calculate the regressionweights (beta_0, beta_1) just like you would with a data point z_i.
For further reference see www.itl.nist.gov/div898/handbook/pmd/section1/dep/dep144.htm
@@stefank4286 BAM! :)
LOESS is a fine-grained sediment, as per Wikipedia or various Scrabble dictionaries if you are into that stuff. Your pronunciation is correct.
Thanks!
I love this channel!!! Thanks for the videos!
Thank you! :)
I'm watching this video because loess and lowess are discussed in statistical method in water resources. Loess and lowess are pronounced the same.
Thanks!
@@statquest do you have a video showing implementation of loess and lowess in R?
@@Geologist997 All I have is this code that demonstrates how to use those functions in R: github.com/StatQuest/lowess_loess_demo/blob/master/lowess_loess_demo.R
Gracias profesor. Usted es el mejor
¡Muchas gracias!
Hi, at 5'34'' when the focal point changes to 6th, why doesn't the window move forward? Thank you!
The window, since it is set to "size = 5", contains the four closest points to the focal point. I explain this at 3:30.
I see. Thank you Josh!
I see. Thank you Josh!
Hello Josh, thanks for this video, it's awesome.
I was just wondering if you had time to finally make a video about splines? Otherwise can you give me links of any documents that can help. I need especialy the part where we plot K+m graphs (K for knots and m for the order). Thanks in advance :)
How could this vid get dislike? this video is awesome
BAM!!! Thank you very much! :)
thanks i was reviewing The Seasonal and Trend decomposition using Loess , your video is fantastic !
Thank you! :)
Could you make a video about fractional polynomials and splines please?great videos.Thanks
I'll put that on the to-do list. :)
@@statquest pleaaaaaaase :)
@@Coucou1981 OK, I've bumped it up a spot on the to-do. :)
How are we choosing the new points here? I am talking about all the "Red Xs". Will we just consider a point on the regression line that is perpendicular to the focal point?
The Red Xs are chosen after fitting a line to the data using weighted least squares. If you would like to learn about least squares, check out this video: ruclips.net/video/PaFPbb66DxQ/видео.html
hmmm this is interesting... i thot curves are fit using the kernel trick every time. this is new information. are you aware of any python library that can perform the same trick. and how does this compare to kernel tricks?
I just did a quick google search and it seems that some people use the statsmodels package to do Lowess curves in python: www.statsmodels.org/dev/generated/statsmodels.nonparametric.smoothers_lowess.lowess.html
thanks for the great content!
Thanks!
Thanks so much for this!!
Will we have more videos on curvilinear fitting? Do p values and inference work the same with curves ? In what way fiiting a curve is better than fitting a line?
Statistics (like p-values) are much harder to calculate for non-linear fits than linear fits.
Thanks ! Keep posting such videos🙏🏻.
Regards :)
That was so helpful! You really break everything down to basics and let the learner soak it in.
Here is something I found on wikipedia: LOESS (LOcally Estimated Scatterplot Smoothing) and LOWESS (LOcally WEighted Scatterplot Smoothing)
nice
Thank you! The first time I've ever seen loess explained clearly.
Hooray! I'm glad the video was helpful. :)
Great video! Recently I've been working with the "Moving Least Square" concept which is widely used in computer graphics, especially image deformation. This video helps me understand this concept in statistical field. Thank you so much.
Could you please upload some videos clearly explaining Non linear methods such as polynomial regression,step function ,splines,Basis function & GAM .
Thanks for making such awesome videos.👌
To quote the Wikipedia article on Polynomial Regression ( en.wikipedia.org/wiki/Polynomial_regression ): "Conveniently, [polynomial regression] models are all linear from the point of view of estimation, since the regression function is linear in terms of the unknown parameters β0, β1, .... Therefore, for least squares analysis, the computational and inferential problems of polynomial regression can be completely addressed using the techniques of multiple regression. This is done by treating x, x2, ... as being distinct independent variables in a multiple regression model." That means you can watch my videos on Linear Regression to learn about Polynomial Regression: ruclips.net/p/PLblh5JKOoLUIzaEkCLIUxQFjPIlapw8nU
excellent way explaining this advance concept ... thanks Josh ...
Thank you! :)
Nice video. I now realise i made something similar with centre moving averages and different (linear/quadratic) best fit per point. As ever i suspected that a sign of a good idea, is that someone else has done it 10x better already.
:)
Great video man! Thanks for sharing.
Thank you! :)
Excellent explanation, clear as crystal. Thank you very much for sharing it.
Thanks! :)
What kind of a curve do you fit to the new-new points? Does the curve have to pass through all the new-new points?
This was a great explanation. Thanks.
Glad it was helpful!
Great video.
Thanks!
Subscribed, many thanks to the information and how you present it.
Hooray! :)
How can a non-parametric LOESS regression be used for extrapolation?
Presumably once you have the curve you can use it to make predictions.
Mindblowing.
bam!
I'm German so I always call it "Löss" :P
Awesome!
This is why I love R. It has all sorts of fun functions like this.
For anyone who is playing with timeseries data, eg sensor data, lowess/loess are smoothing functions and are great for preprocessing and cleaning data. To find out more checkout: en.wikipedia.org/wiki/Smoothing
Bam!!!
:)
Thumbs up just for the intro.
Thanks! :)
Hello, thats so nice of you to teach us the Lowess fitting, I have a specific parameter which needs to be fitted. Yet i need to change the weighing relationship, can you help with some explanation or some material to study for it. Thankyou.
you are a champion. Great video
Maith an buchaill, StatQuest.
Thank you! :)
What is going on with closed captions in the first minute of the video haha
I think you can turn them off.
Thanks a lot for the succinct explanation!!
You're welcome! :)
So, you obtained the new-new points based on the distance of the y axis between the new points and old points? is that so?
What time point, minutes and seconds, are you asking about?
The best theme song so far
Thanks!
So, are these regressions used for better trend visualization only, or there are any other reasons?
You could use them to make predictions with future data. That's essentially what a neural network does (however, it uses a different method to fit lines to the data).
Saved my Time !! Thank you very much
Hooray! :)
what will be a normal window size?default? or do we look at other characteristic to determine? what is the relationship between window size and fit?
I was always told it was pronounced "Low-Ess"
Cool.
Hi Josh, thanks for the video. Does LOWESS allow us to use set of external values to help with smoothing? Let's say I want to smooth some time-series data for one individual in a population. I'd like to do this while keeping in mind the overall population - so, when I smooth this individual, I'm doing it "along with" the population's mean time-series. Is something like this possible with LOWESS? Could I use the population's mean as a some kind of weight?
That's a good and interesting question. Unfortunately I don't know the answer to it.
I am becoming a big StatQuest fan! Congrat for this great video. Just one question... According to what I understand the last points can be misleading because as we get more data, they will change. First we calculate last value of the curve, let's say corresponding to last point X(10), from points on the left, so X(5) to X(10). But as we get new points, let's say X(11), X(12), the value of the curve corresponding to X(10) will change because we now use points X(8) to X(12) in calculation. Am I correct? I hope this is clear... Thanks
Yes, as you add more data, the curve will change.
Hey Josh great video! Any idea why loess (locally estimated scatterplot smoothing) and lowess (locally weighted scatterplot smoothing), are called a local regression ( en.wikipedia.org/wiki/Local_regression ) instead of a local autoregression?
I have no idea.
Thank you so much!
Thanks! :)
Great explanation
Thanks! :)
:)
This lecture was little confusing, how did you make new parabola adding new weight?
What time point, minutes and seconds, are you asking about?
@@statquest from 5:53 sec to 7:00, the question was how did you make that parabola a smooth parabola?
@@sachinrana7554 Presumably you are asking for the formula that takes into account the weights, since I've already provided you with the intuition for what that formula does. Here's the formula: en.wikipedia.org/wiki/Local_regression
Hey Josh,
Is Lowess and Loess the same thing as splines? If not and if you are taking suggestions, I think some content on interpolation or cubic and polynomial splines would be great!
I believe they are different from splines, and I'll keep those topics in mind.
Crystal clear!! Thank you so much!
Hooray! :)
How big should n be for this method?
That depends on your data.
Very well explained as always! Thanks! but i have a little question: when should we use this method? it looks like it overfits the data.
It can be useful when your data has some unknown shape and you want to find the "top" or "bottom" of the shape, rather than just the maximum or minimum values.
Thank you!
You're welcome!
Rafael Irizarry pronounces it the same way in his videos: "low-ess"
Graciasss
De nada. :)
Your videos are amazing! I can finally understand statistics :)
You just earned a sub! ☺
Thank you!!!! I really appreciate it! :)
Is Loess susceptible to outliers?
It depends on how wide the widow is.
I have a probably way too broad question here. I am reading ESL these days. And local linear regression and cubic smoothing spline are throwing around everywhere in the book. But I can't seem to have a good grasp of their relations and differences. What about them vs. neural network? Essentially, all of them are function approximators. Thank you very much!
Yes, they are all used for approximating non-linear functions. In theory Neural Nets are more flexible and can easily approximate more complicated non-linear surfaces. But I don't know all the details.
@@statquest thanks!
Thx！pretty intuitive
You're welcome! :)
i have a question: why does the lowess line look different near the end of my data when i change where my data ends?
Probably because there are different datapoints influencing it.
@@statquest thanks. is there anything that attempts to produce similar results, but only uses data points from the past at each step of the calculation?
@@cw9249 Not that I know of, but I'm no expert in this area....
Fantastic video! thanks a lot.
Thank you for this!! :)
what is the meaning of weight here ?
A weight describes how much a certain factor is concidered in relation to the others. "Importance" is another way to put it.
Thank you for the great explanation of the LOWESS!
Please explain the last iteration, after I have found the first new points you say that I have new weights, can you please explain in depth how do I use them for the next set of points?
Thanks a lot for you deep explanation.
I am learning Data Science and wonder if you are familiar with good statistics courses that will also show me why Normal Distribution has such formula, kernels and other topics like CI's and all I need for ML. Please advise.
Would you be so kind to add boosting algorithms like xgboost? Your explanations are super simple and I would like to get an explanation from you how to use these algorithms.
Excellent!!!
Please do hypothesis testing.
That is coming out in March.
@@statquest Thanks,I think only hypothesis testing and poisson distribution was not included besides that everything is included.
Hello Josh, yes StatQuest is cool but do you have anything about weighted least squares?
Unfortunately I do not. :(
@@statquest please please make one, lol
@@lilyha2470 :)
@@statquest well, when I listen to your videos and go to the class I am way ahead of everyone even the teacher, hahahha
@@lilyha2470 That is totally awesome!!! :)
Lois!
YES!
@@statquest That’s how my marketing statistics instructor used to pronounce it, and that’s how we still pronounce it at the agency where I’m currently working as an analyst! 😁
I'm a masters student and this is helping me a lot with understanding the idea of LOESS. Thank you!
Happy to help!