The FPR50: a simple, but rough, solution to the p values war (?)
HTML-код
- Опубликовано: 7 сен 2024
- This is a recording of a talk given on May 6, 2021, for the UCL Department of Statistical Science. For other stuff on this topic, and links to papers and videos, please go to www.onemol.org....
Abstract
It’s remarkable that statisticians are still at war about how best to decide whether the difference between the means of two independent samples is a result of sampling error alone or whether it’s real.
Most experimenters with no access to professional statistical advice calculate a p values which they then misinterpret as the probability that their results have occurred by chance. Journals continue to advise this procedure. It is to these people that my suggestion is aimed. They will, rightly, continue to ask whether their results have occurred by chance, and the only way to improve practice is to provide them with an alternative that’s simple enough for them to understand.
One simple alternative is to calculate the likelihood ratio, as a measure of the evidence provided by the experiment about the relative plausibility of H0 and H1. This is entirely deductive and frequentist and thus uncontroversial.
If these odds are expressed as a probability, this probability can be interpreted as the posterior probability of H0, for the case where the prior odds are 1, a quantity that I propose should be called the FPR50, the false positive risk when prior P(H1)=0.5. I suggest that this should be cited, along with the p value and confidence interval, to give a better idea of the possible false positive risk.
This is equivalent, in Bayesian terms, to using a prior distribution with the densities concentrated on the null, and on the observed difference. I shall try to justify the use of this simplification, and the use of a skeptical point null hypothesis.
As EJ Wagenmakers said
“At least Bayesians attempt to find an approximate answer to the right question, instead of struggling to interpret an exact answer to the wrong question.”