Replication, Communication, and the Population Dynamics of Scientific Discovery

TitleReplication, Communication, and the Population Dynamics of Scientific Discovery
Publication TypeJournal Article
Year of Publication2015
AuthorsMcElreath, R, Smaldino, PE
Date Published2015/03/10/
Publication Languageeng
AbstractMany published research results are false (Ioannidis 2005), and controversy continues over the roles of replication and publication policy in improving the reliability of research. We develop a mathematical model of scientific discovery in the context of replication, publication bias, and variation in research quality. This model provides a formal framework for reasoning about the normative structure of science. We show that replication may serve as a ratchet that gradually separates true hypotheses from false, but the same factors that make initial findings unreliable also make replications unreliable. The most important factors in improving the reliability of research are the rate of false positives and the base rate of true hypotheses, and we offer suggestions for accomplishing these goals. Our results also clarify recent debates on the communication of replications. Surprisingly, publication bias is not always an obstacle, but instead may have positive impacts---suppression of negative novel findings is often beneficial. We also find that communication of negative replications serves the scientific community even when replicated studies have diminished power. Our model is only a start, but it speaks directly to ongoing debates about the design and conduct of science.