Reproducibility and Replicability in Science


Book Description

One of the pathways by which the scientific community confirms the validity of a new scientific discovery is by repeating the research that produced it. When a scientific effort fails to independently confirm the computations or results of a previous study, some fear that it may be a symptom of a lack of rigor in science, while others argue that such an observed inconsistency can be an important precursor to new discovery. Concerns about reproducibility and replicability have been expressed in both scientific and popular media. As these concerns came to light, Congress requested that the National Academies of Sciences, Engineering, and Medicine conduct a study to assess the extent of issues related to reproducibility and replicability and to offer recommendations for improving rigor and transparency in scientific research. Reproducibility and Replicability in Science defines reproducibility and replicability and examines the factors that may lead to non-reproducibility and non-replicability in research. Unlike the typical expectation of reproducibility between two computations, expectations about replicability are more nuanced, and in some cases a lack of replicability can aid the process of scientific discovery. This report provides recommendations to researchers, academic institutions, journals, and funders on steps they can take to improve reproducibility and replicability in science.




Introduction to the Responsible Conduct of Research (rev. Ed. )


Book Description

This report seeks to supplement existing resources by making a comprehensive overview of basic rules of the road for responsible research available to all U.S. Public Health Service-funded researchers. It has been prepared with the needs of small and mid-size research and institutions and beginning researchers in mind, but it may be used in other settings. Illustrations.




Reproducible Research with R and RStudio


Book Description

Praise for previous editions: "Gandrud has written a great outline of how a fully reproducible research project should look from start to finish, with brief explanations of each tool that he uses along the way... Advanced undergraduate students in mathematics, statistics, and similar fields as well as students just beginning their graduate studies would benefit the most from reading this book. Many more experienced R users or second-year graduate students might find themselves thinking, ‘I wish I’d read this book at the start of my studies, when I was first learning R!’...This book could be used as the main text for a class on reproducible research ..." (The American Statistician) Reproducible Research with R and R Studio, Third Edition brings together the skills and tools needed for doing and presenting computational research. Using straightforward examples, the book takes you through an entire reproducible research workflow. This practical workflow enables you to gather and analyze data as well as dynamically present results in print and on the web. Supplementary materials and example are available on the author’s website. New to the Third Edition Updated package recommendations, examples, URLs, and removed technologies no longer in regular use. More advanced R Markdown (and less LaTeX) in discussions of markup languages and examples. Stronger focus on reproducible working directory tools. Updated discussion of cloud storage services and persistent reproducible material citation. Added discussion of Jupyter notebooks and reproducible practices in industry. Examples of data manipulation with Tidyverse tibbles (in addition to standard data frames) and pivot_longer() and pivot_wider() functions for pivoting data. Features Incorporates the most important advances that have been developed since the editions were published Describes a complete reproducible research workflow, from data gathering to the presentation of results Shows how to automatically generate tables and figures using R Includes instructions on formatting a presentation document via markup languages Discusses cloud storage and versioning services, particularly Github Explains how to use Unix-like shell programs for working with large research projects




The Practice of Reproducible Research


Book Description

The Practice of Reproducible Research presents concrete examples of how researchers in the data-intensive sciences are working to improve the reproducibility of their research projects. In each of the thirty-one case studies in this volume, the author or team describes the workflow that they used to complete a real-world research project. Authors highlight how they utilized particular tools, ideas, and practices to support reproducibility, emphasizing the very practical how, rather than the why or what, of conducting reproducible research. Part 1 provides an accessible introduction to reproducible research, a basic reproducible research project template, and a synthesis of lessons learned from across the thirty-one case studies. Parts 2 and 3 focus on the case studies themselves. The Practice of Reproducible Research is an invaluable resource for students and researchers who wish to better understand the practice of data-intensive sciences and learn how to make their own research more reproducible.




Good Research Practice in Non-Clinical Pharmacology and Biomedicine


Book Description

This open access book, published under a CC BY 4.0 license in the Pubmed indexed book series Handbook of Experimental Pharmacology, provides up-to-date information on best practice to improve experimental design and quality of research in non-clinical pharmacology and biomedicine.




Fostering Integrity in Research


Book Description

The integrity of knowledge that emerges from research is based on individual and collective adherence to core values of objectivity, honesty, openness, fairness, accountability, and stewardship. Integrity in science means that the organizations in which research is conducted encourage those involved to exemplify these values in every step of the research process. Understanding the dynamics that support â€" or distort â€" practices that uphold the integrity of research by all participants ensures that the research enterprise advances knowledge. The 1992 report Responsible Science: Ensuring the Integrity of the Research Process evaluated issues related to scientific responsibility and the conduct of research. It provided a valuable service in describing and analyzing a very complicated set of issues, and has served as a crucial basis for thinking about research integrity for more than two decades. However, as experience has accumulated with various forms of research misconduct, detrimental research practices, and other forms of misconduct, as subsequent empirical research has revealed more about the nature of scientific misconduct, and because technological and social changes have altered the environment in which science is conducted, it is clear that the framework established more than two decades ago needs to be updated. Responsible Science served as a valuable benchmark to set the context for this most recent analysis and to help guide the committee's thought process. Fostering Integrity in Research identifies best practices in research and recommends practical options for discouraging and addressing research misconduct and detrimental research practices.




Implementing Reproducible Research


Book Description

In computational science, reproducibility requires that researchers make code and data available to others so that the data can be analyzed in a similar manner as in the original publication. Code must be available to be distributed, data must be accessible in a readable format, and a platform must be available for widely distributing the data and code. In addition, both data and code need to be licensed permissively enough so that others can reproduce the work without a substantial legal burden. Implementing Reproducible Research covers many of the elements necessary for conducting and distributing reproducible research. It explains how to accurately reproduce a scientific result. Divided into three parts, the book discusses the tools, practices, and dissemination platforms for ensuring reproducibility in computational science. It describes: Computational tools, such as Sweave, knitr, VisTrails, Sumatra, CDE, and the Declaratron system Open source practices, good programming practices, trends in open science, and the role of cloud computing in reproducible research Software and methodological platforms, including open source software packages, RunMyCode platform, and open access journals Each part presents contributions from leaders who have developed software and other products that have advanced the field. Supplementary material is available at www.ImplementingRR.org.




Statistical Challenges in Assessing and Fostering the Reproducibility of Scientific Results


Book Description

Questions about the reproducibility of scientific research have been raised in numerous settings and have gained visibility through several high-profile journal and popular press articles. Quantitative issues contributing to reproducibility challenges have been considered (including improper data measurement and analysis, inadequate statistical expertise, and incomplete data, among others), but there is no clear consensus on how best to approach or to minimize these problems. A lack of reproducibility of scientific results has created some distrust in scientific findings among the general public, scientists, funding agencies, and industries. While studies fail for a variety of reasons, many factors contribute to the lack of perfect reproducibility, including insufficient training in experimental design, misaligned incentives for publication and the implications for university tenure, intentional manipulation, poor data management and analysis, and inadequate instances of statistical inference. The workshop summarized in this report was designed not to address the social and experimental challenges but instead to focus on the latter issues of improper data management and analysis, inadequate statistical expertise, incomplete data, and difficulties applying sound statistic inference to the available data. Many efforts have emerged over recent years to draw attention to and improve reproducibility of scientific work. This report uniquely focuses on the statistical perspective of three issues: the extent of reproducibility, the causes of reproducibility failures, and the potential remedies for these failures.




Rigor Mortis


Book Description

An essential book to understanding whether the new miracle cure is good science or simply too good to be true American taxpayers spend $30 billion annually funding biomedical research, but over half of these studies can't be replicated due to poor experimental design, improper methods, and sloppy statistics. Bad science doesn't just hold back medical progress, it can sign the equivalent of a death sentence for terminal patients. In Rigor Mortis, Richard Harris explores these urgent issues with vivid anecdotes, personal stories, and interviews with the top biomedical researchers. We need to fix our dysfunctional biomedical system -- before it's too late.




Guidance for the Description of Animal Research in Scientific Publications


Book Description

The publication of research articles involving animal studies is central to many disciplines in science and biomedicine. Effective descriptions in such publications enable researchers to interpret the data, evaluate and replicate findings, and move the science forward. Analyses of published studies with research animals have demonstrated numerous deficiencies in the reporting of details in research methods for animal studies. Considerable variation in the amount of information required by scientific publications and reported by authors undermines this basic scientific principle and results in the unnecessary use of animals and other resources in failed efforts to reproduce study results. Guidance for the Description of Animal Research in Scientific Publications outlines the information that should be included in scientific papers regarding the animal studies to ensure that the study can be replicated. The report urges journal editors to actively promote effective and ethical research by encouraging the provision of sufficient information. Examples of this information include: conditions of housing and husbandry, genetic nomenclature, microbial status, detailed experimental manipulations, and handling and use of pharmaceuticals. Inclusion of this information will enable assessment and interpretation of research findings and advancement of knowledge based on reproducible results.