Most lectures in Statistics are merely a series of steps enunciated ,but yours stands out as something which provokes critical thinking and initiates great Insight!
Great and useful video Daniël. I have a question for you. What should I conclude if I get a significant p- value in my reasearch, for example comparing two groups, but knowing that I had a low power based on a post analysis. Is that result correct? Is there a high probability that result was a false positive instead a valid conclusion? Thanks a lot.
AT 8:29 an approach called "sequential analysis" is recommended. There may be ways to evaluate data 'as it comes in', but the recommendation that was given in this video is not appropriate. If you collect data until you have a statistically significant result, you are P-hacking.
@@DanielLakens I read your paper, and rewatched the part of this video on sequential analysis. I was wrong in the first place about what you were claiming. By "sequential analysis" I had thought you were referring to increasing the sample size until a (conventional) P-value less than alpha is achieved, however that is not what you are claiming in this video or in the psyarxiv.com/9yegd/ paper. Rather you are referring to increasing sample size while also controlling for Type I error. The former is a type of P-hacking, the latter is not. The paper discusses different approaches including decision boundaries, spending functions, conditional power, and Bayesian predictive power. I have not read the sources you cite on these methods to understand their tradeoffs, but an optimal stopping criterion for sample size is desirable if it can be derived.
@@DanielLakens I just refreshed the page and saw the links you responded with. I think my previous comment resolves the misunderstanding, but I may read these later. Thank you.
Most lectures in Statistics are merely a series of steps enunciated ,but yours stands out as something which provokes critical thinking and initiates great Insight!
Great and useful video Daniël. I have a question for you. What should I conclude if I get a significant p- value in my reasearch, for example comparing two groups, but knowing that I had a low power based on a post analysis. Is that result correct? Is there a high probability that result was a false positive instead a valid conclusion? Thanks a lot.
AT 8:29 an approach called "sequential analysis" is recommended. There may be ways to evaluate data 'as it comes in', but the recommendation that was given in this video is not appropriate. If you collect data until you have a statistically significant result, you are P-hacking.
I recommend you look up my statistics papers on sequential analysis to learn why you are wrong. Too many people still just do mindless statistics.
@@DanielLakens I am willing to read one of your papers on sequential analysis. Which one do you think makes the best case for sequential analysis?
@@galenseilis5971 Here you go psyarxiv.com/x4azm/ For a book length treatment link.springer.com/10.1007/978-3-319-32562-0
@@DanielLakens I read your paper, and rewatched the part of this video on sequential analysis. I was wrong in the first place about what you were claiming. By "sequential analysis" I had thought you were referring to increasing the sample size until a (conventional) P-value less than alpha is achieved, however that is not what you are claiming in this video or in the psyarxiv.com/9yegd/ paper. Rather you are referring to increasing sample size while also controlling for Type I error. The former is a type of P-hacking, the latter is not.
The paper discusses different approaches including decision boundaries, spending functions, conditional power, and Bayesian predictive power. I have not read the sources you cite on these methods to understand their tradeoffs, but an optimal stopping criterion for sample size is desirable if it can be derived.
@@DanielLakens I just refreshed the page and saw the links you responded with. I think my previous comment resolves the misunderstanding, but I may read these later. Thank you.