The Shapiro-Wilk test to test for deviations from normality. Also includes an introduction to Q-Q plots, and how they can be used to graphically assess normality.
This was an excellent video, thanks. So, my dataset has columns with very varying counts of values (from 8 to >250). Can I use this test as a rough estimation of normality for each column? Or should I prepare Q-Q plots for each? Also, if I wish to seek the statistical significance between these columns (they measure the same parameter), what test would you recommend? Thanks again.
Remember Shapiro-Wilk's test should not be used for populations greater than 50, so basically it's quite useful for small populations even if it's a bit sensitive. Anyway, great video, maybe you should develop a bit the p-value part explaining how you could interpret it as a confidence indicator about H0.
@@user-hu1sn5or7j the limitation is inheritant to the test method. In their paper, Shapiro and Wilk use a table to compare the W value to a tabulated critical W, but the table goes only from 3 to 50. There is a way to "extend" the test by tabulating yourself the critical values with Monte-Carlo simulation, but that's beyond the Shapiro-Wilk's paper. ;) I don't really know what's behind the scenes of the R calculations but I think that's what is done here.
@@diazjubairy1729 if you compute the critical W values by Monte-Carlo simulation, you may use it with any amount of values but the more data you test the less discriminant it will be, just like every normality test. That's the curse of big samples :')
Наука
Great video! Understood the concept perfectly. A big thanks.
The way you broke this down makes it incredibly easy to understand. thank you!
Thank you for making this video it really clarified the concepts for me
This was an excellent video, thanks.
So, my dataset has columns with very varying counts of values (from 8 to >250).
Can I use this test as a rough estimation of normality for each column? Or should I prepare Q-Q plots for each?
Also, if I wish to seek the statistical significance between these columns (they measure the same parameter), what test would you recommend?
Thanks again.
Remember Shapiro-Wilk's test should not be used for populations greater than 50, so basically it's quite useful for small populations even if it's a bit sensitive.
Anyway, great video, maybe you should develop a bit the p-value part explaining how you could interpret it as a confidence indicator about H0.
Why such a limitation? Which test is best for a population greater than 50?
@@user-hu1sn5or7j the limitation is inheritant to the test method. In their paper, Shapiro and Wilk use a table to compare the W value to a tabulated critical W, but the table goes only from 3 to 50.
There is a way to "extend" the test by tabulating yourself the critical values with Monte-Carlo simulation, but that's beyond the Shapiro-Wilk's paper. ;)
I don't really know what's behind the scenes of the R calculations but I think that's what is done here.
@@krahoc but i've read recent paper talking about shapiro-wilk test is good enough up until 1000 datas
@@diazjubairy1729 if you compute the critical W values by Monte-Carlo simulation, you may use it with any amount of values but the more data you test the less discriminant it will be, just like every normality test. That's the curse of big samples :')
could we get more explanation on the numerator? on the actual derivation of the variable a
Is there a formula to calculate "a"?
Thank you for this Video.. In simple words, explain if the P-value is < 0.05 so Data is normal or Abnormal?
Normal
@@swastikpatro6436 Thank you. ❤
If p < 0.05, the data are not normally distributed. Remember, the null hypothesis for Shapiro.test() is that the data are normally distributed.