Evaluation in the Face of Uncertainty


Book Description

Unexpected events during an evaluation all too often send evaluators into crisis mode. This insightful book provides a systematic framework for diagnosing, anticipating, accommodating, and reining in costs of evaluation surprises. The result is evaluation that is better from a methodological point of view, and more responsive to stakeholders. Jonathan A. Morell identifies the types of surprises that arise at different stages of a program's life cycle and that may affect different aspects of the evaluation, from stakeholder relationships to data quality, methodology, funding, deadlines, information use, and program outcomes. His analysis draws on 18 concise cases from well-known researchers in a variety of evaluation settings. Morell offers guidelines for responding effectively to surprises and for determining the risks and benefits of potential solutions.




Foundation Engineering in the Face of Uncertainty


Book Description

WIDTH: 405pt; BORDER-COLLAPSE: collapse border=0 cellSpacing=0 cellPadding=0 width=540> WIDTH: 405pt; mso-width-source: userset; mso-width-alt: 19748 width=540> HEIGHT: 31.5pt height=42> BORDER-BOTTOM: #f0f0f0; BORDER-LEFT: #f0f0f0; BACKGROUND-COLOR: transparent; WIDTH: 405pt; HEIGHT: 31.5pt; BORDER-TOP: #f0f0f0; BORDER-RIGHT: #f0f0f0 class=xl65 height=42 width=540>GSP 229 contains 54 papers on risk and uncertainty in foundation engineering presented in honor of Fred H. Kulhawy.




The Flaw of Averages


Book Description

A must-read for anyone who makes business decisions that have a major financial impact. As the recent collapse on Wall Street shows, we are often ill-equipped to deal with uncertainty and risk. Yet every day we base our personal and business plans on uncertainties, whether they be next month’s sales, next year’s costs, or tomorrow’s stock price. In The Flaw of Averages, Sam Savageknown for his creative exposition of difficult subjects describes common avoidable mistakes in assessing risk in the face of uncertainty. Along the way, he shows why plans based on average assumptions are wrong, on average, in areas as diverse as healthcare, accounting, the War on Terror, and climate change. In his chapter on Sex and the Central Limit Theorem, he bravely grasps the literary third rail of gender differences. Instead of statistical jargon, Savage presents complex concepts in plain English. In addition, a tightly integrated web site contains numerous animations and simulations to further connect the seat of the reader’s intellect to the seat of their pants. The Flaw of Averages typically results when someone plugs a single number into a spreadsheet to represent an uncertain future quantity. Savage finishes the book with a discussion of the emerging field of Probability Management, which cures this problem though a new technology that can pack thousands of numbers into a single spreadsheet cell. Praise for The Flaw of Averages “Statistical uncertainties are pervasive in decisions we make every day in business, government, and our personal lives. Sam Savage’s lively and engaging book gives any interested reader the insight and the tools to deal effectively with those uncertainties. I highly recommend The Flaw of Averages.” —William J. Perry, Former U.S. Secretary of Defense “Enterprise analysis under uncertainty has long been an academic ideal. . . . In this profound and entertaining book, Professor Savage shows how to make all this practical, practicable, and comprehensible.” —Harry Markowitz, Nobel Laureate in Economics




Decision Making under Deep Uncertainty


Book Description

This open access book focuses on both the theory and practice associated with the tools and approaches for decisionmaking in the face of deep uncertainty. It explores approaches and tools supporting the design of strategic plans under deep uncertainty, and their testing in the real world, including barriers and enablers for their use in practice. The book broadens traditional approaches and tools to include the analysis of actors and networks related to the problem at hand. It also shows how lessons learned in the application process can be used to improve the approaches and tools used in the design process. The book offers guidance in identifying and applying appropriate approaches and tools to design plans, as well as advice on implementing these plans in the real world. For decisionmakers and practitioners, the book includes realistic examples and practical guidelines that should help them understand what decisionmaking under deep uncertainty is and how it may be of assistance to them. Decision Making under Deep Uncertainty: From Theory to Practice is divided into four parts. Part I presents five approaches for designing strategic plans under deep uncertainty: Robust Decision Making, Dynamic Adaptive Planning, Dynamic Adaptive Policy Pathways, Info-Gap Decision Theory, and Engineering Options Analysis. Each approach is worked out in terms of its theoretical foundations, methodological steps to follow when using the approach, latest methodological insights, and challenges for improvement. In Part II, applications of each of these approaches are presented. Based on recent case studies, the practical implications of applying each approach are discussed in depth. Part III focuses on using the approaches and tools in real-world contexts, based on insights from real-world cases. Part IV contains conclusions and a synthesis of the lessons that can be drawn for designing, applying, and implementing strategic plans under deep uncertainty, as well as recommendations for future work. The publication of this book has been funded by the Radboud University, the RAND Corporation, Delft University of Technology, and Deltares.




Changing Bureaucracies


Book Description

In Changing Bureaucracies, international experts provide an unparalleled look at how public sector bureaucracies can better adapt to the reality of unprecedented levels of uncertainty and complexity, and how they can better respond to the emerging needs and demands of citizens and beneficiaries. In particular, they discuss in detail how evaluation can play an important role in aiding bureaucracies in adapting, while noting that the value of evaluation is not at all automatic. Written in a clear and accessible prose, the contributors identify stability as a strength of bureaucratic structures, although adaptability is required in order to remain relevant. They also emphasize the need for bureaucratic rules and practices to be open to examination, such as through evaluation, noting that these rules may take on a life of their own, increasing distrust and conflicting with a meaningful focus on how outcomes and impacts benefit citizens. The book concludes with guidance for both evaluators and for public sector leaders about steps that they can take to improve the responsiveness and relevance of public sector organizations. Pioneering the provision of reflections on how evaluation can play an important role in aiding bureaucracies in adapting, Changing Bureaucracies is an important acquisition for public sector leaders, evaluators, evaluation managers and commissioners and academics alike.




Science and Judgment in Risk Assessment


Book Description

The public depends on competent risk assessment from the federal government and the scientific community to grapple with the threat of pollution. When risk reports turn out to be overblownâ€"or when risks are overlookedâ€"public skepticism abounds. This comprehensive and readable book explores how the U.S. Environmental Protection Agency (EPA) can improve its risk assessment practices, with a focus on implementation of the 1990 Clean Air Act Amendments. With a wealth of detailed information, pertinent examples, and revealing analysis, the volume explores the "default option" and other basic concepts. It offers two views of EPA operations: The first examines how EPA currently assesses exposure to hazardous air pollutants, evaluates the toxicity of a substance, and characterizes the risk to the public. The second, more holistic, view explores how EPA can improve in several critical areas of risk assessment by focusing on cross-cutting themes and incorporating more scientific judgment. This comprehensive volume will be important to the EPA and other agencies, risk managers, environmental advocates, scientists, faculty, students, and concerned individuals.




Environmental Decisions in the Face of Uncertainty


Book Description

The U.S. Environmental Protection Agency (EPA) is one of several federal agencies responsible for protecting Americans against significant risks to human health and the environment. As part of that mission, EPA estimates the nature, magnitude, and likelihood of risks to human health and the environment; identifies the potential regulatory actions that will mitigate those risks and protect public health1 and the environment; and uses that information to decide on appropriate regulatory action. Uncertainties, both qualitative and quantitative, in the data and analyses on which these decisions are based enter into the process at each step. As a result, the informed identification and use of the uncertainties inherent in the process is an essential feature of environmental decision making. EPA requested that the Institute of Medicine (IOM) convene a committee to provide guidance to its decision makers and their partners in states and localities on approaches to managing risk in different contexts when uncertainty is present. It also sought guidance on how information on uncertainty should be presented to help risk managers make sound decisions and to increase transparency in its communications with the public about those decisions. Given that its charge is not limited to human health risk assessment and includes broad questions about managing risks and decision making, in this report the committee examines the analysis of uncertainty in those other areas in addition to human health risks. Environmental Decisions in the Face of Uncertainty explains the statement of task and summarizes the findings of the committee.




The Cunning of Uncertainty


Book Description

Uncertainty is interwoven into human existence. It is a powerful incentive in the search for knowledge and an inherent component of scientific research. We have developed many ways of coping with uncertainty. We make promises, manage risks and make predictions to try to clear the mists and predict ahead. But the future is inherently uncertain - and the mist that shrouds our path an inherent part of our journey. The burning question is whether our societies can face up to uncertainty, learn to embrace it and whether we can open up to a constantly evolving future. In this new book, Helga Nowotny shows how research can thrive at the cusp of uncertainty. Science, she argues, can eventually transform uncertainty into certainty, but into certainty which remains always provisional. Uncertainty is never completely static. It is constantly evolving. It encompasses geological time scales and, at the level of human experience, split-second changes as cells divide. Life and death decisions are taken in the blink of the eye, while human interactions with the natural environment may reveal their impact over millennia. Uncertainty is cunning. It appears at unexpected moments, it shuns the straight line, takes the oblique route and sometimes the unexpected short-cut. As we acknowledge the cunning of uncertainty, its threats retreat. We accept that any scientific inquiry must produce results that are provisional and uncertain. This message is vital for politicians and policy-makers: do not be tempted by small, short-term, controllable gains to the exclusion of uncertain, high-gain opportunities. Wide-ranging in its use of examples and enriched by the author’s experience as President of the European Research Council, one of the world’s leading funding organisations for fundamental research. The Cunning of Uncertainty is a must-read for students and scholars of all disciplines, politicians, policy-makers and anyone concerned with the fundamental role of knowledge and science in our societies today.




Environmental Decisions in the Face of Uncertainty


Book Description

The U.S. Environmental Protection Agency (EPA) is one of several federal agencies responsible for protecting Americans against significant risks to human health and the environment. As part of that mission, EPA estimates the nature, magnitude, and likelihood of risks to human health and the environment; identifies the potential regulatory actions that will mitigate those risks and protect public health1 and the environment; and uses that information to decide on appropriate regulatory action. Uncertainties, both qualitative and quantitative, in the data and analyses on which these decisions are based enter into the process at each step. As a result, the informed identification and use of the uncertainties inherent in the process is an essential feature of environmental decision making. EPA requested that the Institute of Medicine (IOM) convene a committee to provide guidance to its decision makers and their partners in states and localities on approaches to managing risk in different contexts when uncertainty is present. It also sought guidance on how information on uncertainty should be presented to help risk managers make sound decisions and to increase transparency in its communications with the public about those decisions. Given that its charge is not limited to human health risk assessment and includes broad questions about managing risks and decision making, in this report the committee examines the analysis of uncertainty in those other areas in addition to human health risks. Environmental Decisions in the Face of Uncertainty explains the statement of task and summarizes the findings of the committee.




Science and Decisions


Book Description

Risk assessment has become a dominant public policy tool for making choices, based on limited resources, to protect public health and the environment. It has been instrumental to the mission of the U.S. Environmental Protection Agency (EPA) as well as other federal agencies in evaluating public health concerns, informing regulatory and technological decisions, prioritizing research needs and funding, and in developing approaches for cost-benefit analysis. However, risk assessment is at a crossroads. Despite advances in the field, risk assessment faces a number of significant challenges including lengthy delays in making complex decisions; lack of data leading to significant uncertainty in risk assessments; and many chemicals in the marketplace that have not been evaluated and emerging agents requiring assessment. Science and Decisions makes practical scientific and technical recommendations to address these challenges. This book is a complement to the widely used 1983 National Academies book, Risk Assessment in the Federal Government (also known as the Red Book). The earlier book established a framework for the concepts and conduct of risk assessment that has been adopted by numerous expert committees, regulatory agencies, and public health institutions. The new book embeds these concepts within a broader framework for risk-based decision-making. Together, these are essential references for those working in the regulatory and public health fields.