What is a replication crisis and how do I know if I'm in one? Lessons from Psychology

Speaker: Simine Vazire
Affiliation: University of Melbourne

Abstract

The last 15 years have been a turbulent period for the field of psychology. Failures to replicate, fraud, and evidence of questionable research practices have all contributed to a crisis of credibility. What has the field done in response to this crisis, and what lessons can be learned from psychology’s crisis? We often hear the self-correcting mechanisms in science invoked as a reason to trust science, but it is not clear what these mechanisms are, or how they could have prevented psychology’s crisis. Some quality control mechanisms, such as peer review for journals, or vetting for textbooks or for public dissemination, did not to provide much of a safeguard against invalid claims in psychology.  Instead, I argue that we should look for visible signs of a scientific community's commitment to self-correction.  These signs include transparency in the research and peer review process, investment in error detection and quality control, and an emphasis on calibration rather than popularization.  We should trust scientific fields more to the extent that they have these hallmarks of credibility.  Fields that are more transparent, rigorous, and calibrated should earn more trust.