ISBA WM 2021: Leonhard Held: The Statistical Assessment of Replication Success

Keynote Lecture
Speaker
Leonhard Held, University of Zurich
Chair
Marina Vannucci, Rice University, Professor
The Statistical Assessment of Replication Success - A Case for Reverse-Bayes
Replicability of research findings is crucial to the credibility of all empirical domains of science. Large-scale replication projects are increasingly conducted in order to assess to what extent claims of new discoveries can be confirmed in independent replication studies. However, there is no established standard how to assess replication success and in practice many different approaches are used. We argue that Reverse-Bayes methods have a key role to play in the assessment of replication success. The main idea is to reverse Bayes' Theorem to determine a sceptical prior that would make the original finding no longer convincing. Sufficient incompatability of the sceptical prior and the replication study result is then used to quantify the degree of replication success. Two approaches are described in more detail: A Bayes/non-Bayes compromise based on tail probabilities and an alternative approach based on Bayes factors. Case studies from several replication projects show the similarities and differences of the two methods and advantages over alternative approaches.