Replication and Evidence Factors in Observational Studies


Book Description

Outside of randomized experiments, association does not imply causation, and yet there is nothing defective about our knowledge that smoking causes lung cancer, a conclusion reached in the absence of randomized experimentation with humans. How is that possible? If observed associations do not identify causal effects in observational studies, how can a sequence of such associations become decisive? Two or more associations may each be susceptible to unmeasured biases, yet not susceptible to the same biases. An observational study has two evidence factors if it provides two comparisons susceptible to different biases that may be combined as if from independent studies of different data by different investigators, despite using the same data twice. If the two factors concur, then they may exhibit greater insensitivity to unmeasured biases than either factor exhibits on its own. Replication and Evidence Factors in Observational Studies includes four parts: A concise introduction to causal inference, making the book self-contained Practical examples of evidence factors from the health and social sciences with analyses in R The theory of evidence factors Study design with evidence factors A companion R package evident is available from CRAN.




Replication and Evidence Factors in Observational Studies


Book Description

Outside of randomized experiments, association does not imply causation, and yet there is nothing defective about our knowledge that smoking causes lung cancer, a conclusion reached in the absence of randomized experimentation with humans. How is that possible? If observed associations do not identify causal effects in observational studies, how can a sequence of such associations become decisive? Two or more associations may each be susceptible to unmeasured biases, yet not susceptible to the same biases. An observational study has two evidence factors if it provides two comparisons susceptible to different biases that may be combined as if from independent studies of different data by different investigators, despite using the same data twice. If the two factors concur, then they may exhibit greater insensitivity to unmeasured biases than either factor exhibits on its own. Replication and Evidence Factors in Observational studies has four parts: A concise introduction to causal inference, making the book self-contained. Practical examples of evidence factors from the health and social sciences with analyses in R. The theory of evidence factors. Study design with evidence factors. A companion R package evident is available from CRAN.




Evidence Factors for Observational Studies


Book Description

This thesis includes five chapters on evidence factors analysis of causal effect in various observational study settings. Each of these chapters can be read independently without knowledge of the content of any of the other chapters. Evidence factors allow for two independent analyses to be constructed from the same data set. When combining the evidence factors, the type-I error rate must be controlled to obtain valid inference. A powerful method is developed for controlling the familywise error rate for sensitivity analyses to unmeasured confounding with evidence factors. It is shown that the Bahadur efficiency of sensitivity analysis for the combined evidence is greater than for either evidence factor alone. The popular strategy of matching, for controlling the observed covariates, before inferring about the treatment effect, requires solving an optimization problem. This problem can be solved in polynomial time. In an evidence factors analysis we must consider multiple comparisons, thus the matching problem is often of matching at least three groups. This slightly different problem is much more difficult to solve. The third chapter proposes an approximation algorithm to solve this (and more practical versions of this) problem. We prove that the proposed algorithm provides a solution fast, that is provably not a lot further than the optimal solution that is difficult calculate. Two chapters that follow show the applicability of evidence factors analysis in more complicated study designs. The first of these two chapters considers a case-control study with multiple case definitions and the latter one considers studies with instrumental variables, where the instrument(s) may become invalid. The final chapter of the thesis develops a frequentist method for quantification of the degree of corroboration of causal hypothesis using the tool of evidence factors.




Reproducibility and Replicability in Science


Book Description

One of the pathways by which the scientific community confirms the validity of a new scientific discovery is by repeating the research that produced it. When a scientific effort fails to independently confirm the computations or results of a previous study, some fear that it may be a symptom of a lack of rigor in science, while others argue that such an observed inconsistency can be an important precursor to new discovery. Concerns about reproducibility and replicability have been expressed in both scientific and popular media. As these concerns came to light, Congress requested that the National Academies of Sciences, Engineering, and Medicine conduct a study to assess the extent of issues related to reproducibility and replicability and to offer recommendations for improving rigor and transparency in scientific research. Reproducibility and Replicability in Science defines reproducibility and replicability and examines the factors that may lead to non-reproducibility and non-replicability in research. Unlike the typical expectation of reproducibility between two computations, expectations about replicability are more nuanced, and in some cases a lack of replicability can aid the process of scientific discovery. This report provides recommendations to researchers, academic institutions, journals, and funders on steps they can take to improve reproducibility and replicability in science.




A Concise Guide to Observational Studies in Healthcare


Book Description

A Concise Guide to Observational Studies in Healthcare provides busy healthcare professionals with an easy-to-read introduction and overview to conducting, analysing and assessing observational studies. It is a suitable introduction for anyone without prior knowledge of study design, analysis or conduct as the important concepts are presented throughout the text. It provides an overview to the features of design, analyses and conduct of observational studies, without using mathematical formulae, or complex statistics or terminology and is a useful guide for researchers conducting their own studies, those who participate in studies co-ordinated by others, or who read or review a published report of an observational study. Examples are based on clinical features of people, biomarkers, lifestyle habits and environmental exposures, and evaluating quality of care.




Causal Inference


Book Description

A nontechnical guide to the basic ideas of modern causal inference, with illustrations from health, the economy, and public policy. Which of two antiviral drugs does the most to save people infected with Ebola virus? Does a daily glass of wine prolong or shorten life? Does winning the lottery make you more or less likely to go bankrupt? How do you identify genes that cause disease? Do unions raise wages? Do some antibiotics have lethal side effects? Does the Earned Income Tax Credit help people enter the workforce? Causal Inference provides a brief and nontechnical introduction to randomized experiments, propensity scores, natural experiments, instrumental variables, sensitivity analysis, and quasi-experimental devices. Ideas are illustrated with examples from medicine, epidemiology, economics and business, the social sciences, and public policy.




Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide


Book Description

This User’s Guide is a resource for investigators and stakeholders who develop and review observational comparative effectiveness research protocols. It explains how to (1) identify key considerations and best practices for research design; (2) build a protocol based on these standards and best practices; and (3) judge the adequacy and completeness of a protocol. Eleven chapters cover all aspects of research design, including: developing study objectives, defining and refining study questions, addressing the heterogeneity of treatment effect, characterizing exposure, selecting a comparator, defining and measuring outcomes, and identifying optimal data sources. Checklists of guidance and key considerations for protocols are provided at the end of each chapter. The User’s Guide was created by researchers affiliated with AHRQ’s Effective Health Care Program, particularly those who participated in AHRQ’s DEcIDE (Developing Evidence to Inform Decisions About Effectiveness) program. Chapters were subject to multiple internal and external independent reviews. More more information, please consult the Agency website: www.effectivehealthcare.ahrq.gov)




Design of Observational Studies


Book Description

An observational study is an empiric investigation of effects caused by treatments when randomized experimentation is unethical or infeasible. Observational studies are common in most fields that study the effects of treatments on people, including medicine, economics, epidemiology, education, psychology, political science and sociology. The quality and strength of evidence provided by an observational study is determined largely by its design. Design of Observational Studies is both an introduction to statistical inference in observational studies and a detailed discussion of the principles that guide the design of observational studies. Design of Observational Studies is divided into four parts. Chapters 2, 3, and 5 of Part I cover concisely, in about one hundred pages, many of the ideas discussed in Rosenbaum’s Observational Studies (also published by Springer) but in a less technical fashion. Part II discusses the practical aspects of using propensity scores and other tools to create a matched comparison that balances many covariates. Part II includes a chapter on matching in R. In Part III, the concept of design sensitivity is used to appraise the relative ability of competing designs to distinguish treatment effects from biases due to unmeasured covariates. Part IV discusses planning the analysis of an observational study, with particular reference to Sir Ronald Fisher’s striking advice for observational studies, "make your theories elaborate." The second edition of his book, Observational Studies, was published by Springer in 2002.




The Production of Knowledge


Book Description

A wide-ranging discussion of factors that impede the cumulation of knowledge in the social sciences, including problems of transparency, replication, and reliability. Rather than focusing on individual studies or methods, this book examines how collective institutions and practices have (often unintended) impacts on the production of knowledge.




Observational Studies in a Learning Health System


Book Description

Clinical research strains to keep up with the rapid and iterative evolution of medical interventions, clinical practice innovation, and the increasing demand for information on the clinical effectiveness of these advancements. In response to the growing availability of archived and real-time digital health data and the opportunities this data provides for research, as well as the increasing number of studies using prospectively collected clinical data, the Institute of Medicine\'s Roundtable on Value & Science-Driven Health Care convened a workshop on Observational Studies in a Learning Health System. Participants, including experts from a wide range of disciplines - clinical researchers, statisticians, biostatisticians, epidemiologists, health care informaticians, health care analytics, research funders, health products industry, clinicians, payers, and regulators - explored leading edge approaches to observational studies, charted a course for the use of the growing health data utility, and identified opportunities to advance progress. Workshop speakers and individual participants strove to identify stakeholder needs and barriers to the broader application of observational studies. Observational Studies in a Learning Health Systemis the summary of the workshop. This report explores the role of observational studies in the generation of evidence to guide clinical and health policy decisions. The report discusses concepts of rigorous observational study design and analysis, emerging statistical methods, and opportunities and challenges of observational studies to complement evidence from experimental methods, treatment heterogeneity, and effectiveness estimates tailored toward individual patients.