Full course is now available on my private website. Become a member and get full access: meerkatstatistics.com/courses... * 🎉 Special RUclips 60% Discount on Yearly Plan - valid for the 1st 100 subscribers; Voucher code: First100 🎉 * “GLM in R” Course Outline: Administration * Administration Up to Scratch * Notebook - Introduction * Notebook - Linear Models * Notebook - Intro to R Intro to GLM’s * Linear Models vs. Generalized Linear Models * Least Squares vs. Maximum Likelihood * Saturated vs. Constrained Model * Link Functions Exponential Family * Definition and Examples * More Examples * Notebook - Exponential Family * Mean and Variance * Notebook - Mean-Variance Relationship Deviance * Deviance * Notebook - Deviance Likelihood Analysis * Likelihood Analysis * Numerical Solution * Notebook - GLM’s in R * Notebook - Fitting the GLM * Inference Code Examples: * Notebook - Binary/Binomial Regression * Notebook - Poisson & Negative Binomial Regression * Notebook - Gamma & Inverse Gaussian Regression Advanced Topics: * Quasi-Likelihood * Generalized Estimating Equations (GEE) * Mixed Models (GLMM) Why become a member? * All video content * Extra material (notebooks) * Access to code and notes * Community Discussion * No Ads * Support the Creator ❤
thanks a lot for the explanation. But I was thinking that in the linear model, Normally we use Y as the response variable and X as the independent variable, where response Y is dependent on X. That's why I got a little bit confused at first when u are taking Y as the observations.
Independent variable Y doesn’t need to be Normally distributed, it just need to be a from distribution from Exponential Family. The only assumed Normality is the Residual
Gauss didn't invent the linear model; he just claimed to a decade after someone else had. The same is true for Gaussian elimination. Newton invented it, and then Gauss decided to name it after himself.
I'm trying to wrap my head around your point about the second assumption here - I thought with regular linear models it was required that the _residuals_ be normally distributed, not the data points themselves, but then is it that the residuals in GLMs can be from non-normal distributions? (as long as they're in the exponential family)
y=bx+residual, i.e. a systematic part + a stochastic part. Hence if the residuals are normal, the y's are normal. Or you could say, the y's are normal because the noise is normal.
@@MeerkatStatistics I think I understand @Tul Lee comment. IIUC it is the same as my doubt here. Let me use a great example I saw somewhere else to clarify this: The house prices in a city are linearly related to its area (in square feet/meters). Now, suppose I observe the prices and areas of some houses, and I noticed that there are more or less the same number of houses (let us say, 50±3) in each decile of the observed prices and observed areas. My price data points are uniformly distributed, yet I still can use linear regression. So, normally distributed prices are not a prerequisite for having a linear model, am I right? What is a prerequisite is that the *residuals* (defined as R = Y-bX) are normally distributed, R ~ N(0, σ²). Did we interpret it wrong?
@@arnbrandy If y is uniform, the residuals are uniform. In this model there is only 1 source of stochasticity which are the residuals. They can be either uniform or normal but not both. So yes, you are wrong. However - there is no real requirement for the distribution to be normal. As mentioned in another comment here, it's actually a subclass of linear models called Normal Linear Models, which make some results (CI for the coefficients, Prediction Intervals, etc.) easier to get (though you can get them using asymptotic theory if n is large enough). The only thing "required" is for the residuals to have 0 mean and constant independent variance (though even non constant and non independent variance can be dealt with using GLS). This is just an introduction to the topic which tries to simplify it in order for most people to grasp it. Like most simplifications, it does not capture the full complexity and subtleties of the topic.
@@MeerkatStatistics This assumptions are for the residuals and not for the y_i. He is correct, there are no restriction or assumptions about the observed distribution itself.
@@hanshansi7597 For "simple" (fixed covariates x) regression you can assume normality for the residuals, or you can assume it for the y's. Since in this case b and x are considered fixed, the only source of randomness for y is the epsilon. If you have random x's, then the story differs.
Hi! Awesome videos dude! I do have a few questions - Are all linear models, gaussian linear models ? The assumption that errors/residuals have to be normally distributed, does it hold true for both regular linear models as well as GLMs ? Why cannot be use LSE for GLMs ?
At 2:11 I don't understand what you mean be "the coefficients are made linear". If we have a term like beta_i^2, what prevents us from redefining beta_i^2 -> beta_i? The beta's are all c-numbers, right?
Nothing prevents you. But if you have y=beta0+beta1*x1 + x2^beta2, that's a problem. Same if you have y=beta1*x1/(beta0+beta2*x2). The main point is that the function has to be linear w.r.t. to the inputs, but it's ok if the x's are some transformations of themselves (i.e. x^2, log x, exp x, sin x, etc.).
Hi, great video 👍🏻 Just a comment/question. Perhaps am I wrong, but in my understanding, the normality of residuals (or of y's if you prefer) is not formally required for the parameter estimation for the best fit line using the ordinary least square methods (linear model). The Gauss-Markov theorem requires other assumptions about the errors (such as finite variance / homoscedasticity or zero conditional mean...) to ensure that the OLS gives the best linear estimator... but normality itself is mostly important for inference (drawing confidence intervals), not for parameter estimate. In other words, even in violation of normality, we cannot conclude that the OLS would not give the best linear unbiased estimator. As I said, perhaps am I wrong.
No, I think you are right. There are some properties that can be achieved by simply demanding homoscedasticity or zero conditional mean. But for more properties you will need also the normal assumption. In Agresti's book (Foundation of Linear and Generalized Linear models) chapter 2 is devoted to Linear models without the assumption of normality, and chapter 3 is devoted to "Normal Linear Models". You should check it out.
They are now offered for paid members in my website: meerkatstatistics.com/courses/generalized-linear-models-glms/ I made a video explaining how to register ruclips.net/video/JUNO4wIVoo0/видео.html&ab_channel=MeerkatStatistics
Thank you so much! I have a question - how is the ‘generalized least squares’ and ‘general linear models’ categorized wrt. to these two, and what are their differences to these respectively?
GLS is a different concept - used in linear models (linear regression). It accounts for a residual covariance matrix which is not the identity (i.e., the assumption of homoscedasticity and independence are violated). I might do a video about it in the future.
Full course is now available on my private website. Become a member and get full access:
meerkatstatistics.com/courses...
* 🎉 Special RUclips 60% Discount on Yearly Plan - valid for the 1st 100 subscribers; Voucher code: First100 🎉 *
“GLM in R” Course Outline:
Administration
* Administration
Up to Scratch
* Notebook - Introduction
* Notebook - Linear Models
* Notebook - Intro to R
Intro to GLM’s
* Linear Models vs. Generalized Linear Models
* Least Squares vs. Maximum Likelihood
* Saturated vs. Constrained Model
* Link Functions
Exponential Family
* Definition and Examples
* More Examples
* Notebook - Exponential Family
* Mean and Variance
* Notebook - Mean-Variance Relationship
Deviance
* Deviance
* Notebook - Deviance
Likelihood Analysis
* Likelihood Analysis
* Numerical Solution
* Notebook - GLM’s in R
* Notebook - Fitting the GLM
* Inference
Code Examples:
* Notebook - Binary/Binomial Regression
* Notebook - Poisson & Negative Binomial Regression
* Notebook - Gamma & Inverse Gaussian Regression
Advanced Topics:
* Quasi-Likelihood
* Generalized Estimating Equations (GEE)
* Mixed Models (GLMM)
Why become a member?
* All video content
* Extra material (notebooks)
* Access to code and notes
* Community Discussion
* No Ads
* Support the Creator ❤
I tried to sign up but it wouldn't work
This is the best high level explanation yet to understand the motivation.
Great video. Just the explanation I was looking for!
thanks a lot for the explanation. But I was thinking that in the linear model,
Normally we use Y as the response variable and X as the independent variable, where response Y is dependent on X.
That's why I got a little bit confused at first when u are taking Y as the observations.
Independent variable Y doesn’t need to be Normally distributed, it just need to be a from distribution from Exponential Family. The only assumed Normality is the Residual
In linear models, y is normal. In GLM's y can be any exponential family.
@@MeerkatStatistics y should be normal given x
excelent, thank you
Thankss a lot...this was very helpful.😀
The Xi are indépendante not yi
thx!!
Gauss didn't invent the linear model; he just claimed to a decade after someone else had. The same is true for Gaussian elimination. Newton invented it, and then Gauss decided to name it after himself.
wow! thanks very much! big shout out from brazil
I'm trying to wrap my head around your point about the second assumption here - I thought with regular linear models it was required that the _residuals_ be normally distributed, not the data points themselves, but then is it that the residuals in GLMs can be from non-normal distributions? (as long as they're in the exponential family)
y=bx+residual, i.e. a systematic part + a stochastic part. Hence if the residuals are normal, the y's are normal. Or you could say, the y's are normal because the noise is normal.
@@MeerkatStatistics in this case, y can be uniformed and still maintaining Normal residual
@@tullee7228 not sure I understand what you mean. Uniform and Normal are two different distribution, and a random variable can't be both.
@@MeerkatStatistics I think I understand @Tul Lee comment. IIUC it is the same as my doubt here. Let me use a great example I saw somewhere else to clarify this:
The house prices in a city are linearly related to its area (in square feet/meters). Now, suppose I observe the prices and areas of some houses, and I noticed that there are more or less the same number of houses (let us say, 50±3) in each decile of the observed prices and observed areas. My price data points are uniformly distributed, yet I still can use linear regression. So, normally distributed prices are not a prerequisite for having a linear model, am I right?
What is a prerequisite is that the *residuals* (defined as R = Y-bX) are normally distributed, R ~ N(0, σ²).
Did we interpret it wrong?
@@arnbrandy If y is uniform, the residuals are uniform. In this model there is only 1 source of stochasticity which are the residuals. They can be either uniform or normal but not both. So yes, you are wrong.
However - there is no real requirement for the distribution to be normal. As mentioned in another comment here, it's actually a subclass of linear models called Normal Linear Models, which make some results (CI for the coefficients, Prediction Intervals, etc.) easier to get (though you can get them using asymptotic theory if n is large enough). The only thing "required" is for the residuals to have 0 mean and constant independent variance (though even non constant and non independent variance can be dealt with using GLS).
This is just an introduction to the topic which tries to simplify it in order for most people to grasp it. Like most simplifications, it does not capture the full complexity and subtleties of the topic.
Thank you. What's the difference between MLE and Least squares? Sorry for the stupid question.
See the next video in the series 🙂
Hi! Thanks for the video. Can you please explain (in comments or in a video) how this relates to GLS?
They are two different things. Check here stats.stackexchange.com/a/272562/117705
Linear regression assumptions are wrong. -Every observation doesn’t need to be normal. Residuals need to follow a normal distribution
If the residual (epsilon) is normal, what does it mean about the observation (y)? When y=b*x + epsilon.
@@MeerkatStatistics This assumptions are for the residuals and not for the y_i. He is correct, there are no restriction or assumptions about the observed distribution itself.
@@hanshansi7597 For "simple" (fixed covariates x) regression you can assume normality for the residuals, or you can assume it for the y's. Since in this case b and x are considered fixed, the only source of randomness for y is the epsilon.
If you have random x's, then the story differs.
omg, finally i found short and clear video , thank u !
Hard to understand.
Hi! Awesome videos dude! I do have a few questions -
Are all linear models, gaussian linear models ?
The assumption that errors/residuals have to be normally distributed, does it hold true for both regular linear models as well as GLMs ?
Why cannot be use LSE for GLMs ?
At 2:11 I don't understand what you mean be "the coefficients are made linear". If we have a term like beta_i^2, what prevents us from redefining beta_i^2 -> beta_i? The beta's are all c-numbers, right?
Nothing prevents you. But if you have y=beta0+beta1*x1 + x2^beta2, that's a problem. Same if you have y=beta1*x1/(beta0+beta2*x2). The main point is that the function has to be linear w.r.t. to the inputs, but it's ok if the x's are some transformations of themselves (i.e. x^2, log x, exp x, sin x, etc.).
This was awesome dude! Thanks for that, really!
Warms my heart :-)
y does not need to be normally distributed
Hi, great video 👍🏻 Just a comment/question. Perhaps am I wrong, but in my understanding, the normality of residuals (or of y's if you prefer) is not formally required for the parameter estimation for the best fit line using the ordinary least square methods (linear model). The Gauss-Markov theorem requires other assumptions about the errors (such as finite variance / homoscedasticity or zero conditional mean...) to ensure that the OLS gives the best linear estimator... but normality itself is mostly important for inference (drawing confidence intervals), not for parameter estimate. In other words, even in violation of normality, we cannot conclude that the OLS would not give the best linear unbiased estimator. As I said, perhaps am I wrong.
No, I think you are right. There are some properties that can be achieved by simply demanding homoscedasticity or zero conditional mean. But for more properties you will need also the normal assumption. In Agresti's book (Foundation of Linear and Generalized Linear models) chapter 2 is devoted to Linear models without the assumption of normality, and chapter 3 is devoted to "Normal Linear Models". You should check it out.
@@MeerkatStatistics Thanks for the answer and for the tip, I'll have a look indeed :)
It is the best lecture that I have watched on RUclips. Thanks.
very well explained
Fantastic! Bravo!
YOU ARE AWESOME
Thanks, I love this video so much
Why are the videos hidden?
How can i get them??
They are now offered for paid members in my website: meerkatstatistics.com/courses/generalized-linear-models-glms/
I made a video explaining how to register ruclips.net/video/JUNO4wIVoo0/видео.html&ab_channel=MeerkatStatistics
thanka man
Stupendous!
Thank you
Thanks, that's very helpful :)
Very Clear. Thank you.
good explanation, keep on
thank you!
Thanks!!
this was great! what is the app you use for writing? is it a white board?
Yes, but I since moved to OneNote
Thank you so much! I have a question - how is the ‘generalized least squares’ and ‘general linear models’ categorized wrt. to these two, and what are their differences to these respectively?
GLS is a different concept - used in linear models (linear regression). It accounts for a residual covariance matrix which is not the identity (i.e., the assumption of homoscedasticity and independence are violated). I might do a video about it in the future.