Book Description

Science is allegedly in the midst of a reproducibility crisis, but questions of reproducibility and related principles date back nearly 80 years. Numerous controversies have arisen, especially since 2010, in a wide array of disciplines that stem from the failure to reproduce studies or their findings:biology, biomedical and preclinical research, business and organizational studies, computational sciences, drug discovery, economics, education, epidemiology and statistics, genetics, immunology, policy research, political science, psychology, and sociology. This monograph defines terms and constructs related to reproducible research, weighs key considerations and challenges in reproducing or replicating studies, and discusses transparency in publications that can support reproducible research goals. It attempts to clarify reproducible research, with its attendant (and confusing or even conflicting) lexicon and aims to provide useful background, definitions, and practical guidance for all readers. Among its conclusions: First, researchers must become better educated about these issues, particularly the differences between the concepts and terms. The main benefit is being able to communicate clearly within their own fields and, more importantly, across multiple disciplines. In addition, scientists need to embrace these concepts as part of their responsibilities as good stewards of research funding and as providers of credible information for policy decision making across many areas of public concern. Finally, although focusing on transparency and documentation is essential, ultimately the goal is achieving the most rigorous, high-quality science possible given limitations on time, funding, or other resources. “The authors have written a nuanced and thoughtful primer on scientific reproducibility. By highlighting the social, political, and technical importance of reproducibility, together with a precise description of the related concepts of reproducibility, replicability, and repeatability, this primer provides a significant resource that all practicing researchers should read.” Daniel Reed, Vice President for Research and Economic Development, University of Iowa and former Corporate Vice President, Microsoft “This is a well-written, clearly articulated, and timely primer on the developing and evolving rich terminology of reproducible research. The primer, put together by authors with deep experience and expertise in the topic area, focuses primarily on human-centric research in biomedicine, medicine, and the social sciences as well as reproducibility issues in analytics and computational science. The growing focus on reproducibility will open new vistas in research methodologies, meta analysis, comparative studies of research results, and reuse and adaptation of results from prior research. This primer provides an excellent overview of the subject area, and I would recommend it to anyone interested in coming up to speed on current issues in reproducible research.” Chaitan Baru, Distinguished Scientist and Associate Director for Data Initiatives, San Diego Supercomputing Center; current appointment as Senior Advisor for Data Science, Computer and Information Science and Engineering Directorate, National Science Foundation “Pellizzari et al. have taken on the Herculean task of collecting, synthesizing, and relating the various interpretations of reproducibility used in the research community today, and turned the result into an accessible must-read guide. This important work provides a Rosetta Stone for various stakeholders to discuss and implement solutions that make real progress toward a research enterprise that routinely produces reproducible findings.” Victoria Stodden, Associate Professor at the School of Information Sciences, University of Illinois at Urbana Champaign and co-editor of the books Implementing Reproducible Research and Privacy, Big Data, and the Public Good: Frameworks for Engagement