Program Evaluation


Book Description

"Amid efforts to improve performance and constrain spending, federal agencies are being asked to expand the use of rigorous program evaluation in decision-making. In addition to performance data, indepth program evaluation studies are often needed for assessing program impact or designing improvements. Agencies can also use their evaluation resources to provide information needed for effective management and legislative oversight.GAO was asked to study federal agencies with mature evaluation capacity to examine (1) the criteria, policies, and procedures they use to determine programs to review, and (2) the influences on their choices. GAO reviewed agency materials and interviewed officials on evaluation planning in four agencies in three departments with extensive evaluation experience: Education, Health and Human Services (HHS), and Housing and Urban Development (HUD).HHS and HUD agreed with the description of how they plan evaluations. HHS noted that the optimal location of evaluation units will vary with the circumstances and purpose of evaluations. HUD felt the draft report did not emphasize enough the influence of the appropriations process. GAO has added text to note its influence..."




Evaluating Federal Research Programs


Book Description

The Government Performance and Results Act (GPRA), passed by Congress in 1993, requires that federal agencies write five-year strategic plans with annual performance goals and produce an annual report that demonstrates whether the goals have been met. The first performance reports are due in March 2000. Measuring the performance of basic research is particularly challenging because major breakthroughs can be unpredictable and difficult to assess in the short term. This book recommends that federal agencies use an "expert review" method to examine the quality of research they support, the relevance of that research to their mission, and whether the research is at the international forefront of scientific and technological knowledge. It also addresses the issues of matching evaluation measurements to the character of the research performed, improving coordination among agencies when research is in the same field, and including a human resource development component in GPRA strategic and performance plans.




Program Evaluation


Book Description







Program Evaluation


Book Description

Amid efforts to improve performance and constrain spending, federal agencies are being asked to expand the use of rigorous program evaluation in decision-making. In addition to performance data, indepth program evaluation studies are often needed for assessing program impact or designing improvements. Agencies can also use their evaluation resources to provide information needed for effective management and legislative oversight. GAO (Government Accountability Office) was asked to study federal agencies with mature evaluation capacity to examine (1) the criteria, policies, and procedures they use to determine programs to review, and (2) the influences on their choices. GAO reviewed agency materials and interviewed officials on evaluation planning in four agencies in three departments with extensive evaluation experience: Education, Health and Human Services (HHS), and Housing and Urban Development (HUD). HHS and HUD agreed with the description of how they plan evaluations. HHS noted that the optimal location of evaluation units will vary with the circumstances and purpose of evaluations. HUD felt the draft report did not emphasize enough the influence of the appropriations process. GAO has added text to note its influence on evaluation planning. Education provided technical comments. GAO makes no recommendations. Appendices include: (1) Comments from the Department of Health and Human Services; (2) Comments from the Department of Housing and Urban Development; and (3) GAO Contact and Staff Acknowledgments. A bibliography is included. (Contains 1 figure and 12 footnotes.).




The National Artificial Intelligence Research and Development Strategic Plan


Book Description

Artificial intelligence (AI) is a transformative technology that holds promise for tremendous societal and economic benefit. AI has the potential to revolutionize how we live, work, learn, discover, and communicate. AI research can further our national priorities, including increased economic prosperity, improved educational opportunities and quality of life, and enhanced national and homeland security. Because of these potential benefits, the U.S. government has invested in AI research for many years. Yet, as with any significant technology in which the Federal government has interest, there are not only tremendous opportunities but also a number of considerations that must be taken into account in guiding the overall direction of Federally-funded R&D in AI. On May 3, 2016, the Administration announced the formation of a new NSTC Subcommittee on Machine Learning and Artificial intelligence, to help coordinate Federal activity in AI.1 This Subcommittee, on June 15, 2016, directed the Subcommittee on Networking and Information Technology Research and Development (NITRD) to create a National Artificial Intelligence Research and Development Strategic Plan. A NITRD Task Force on Artificial Intelligence was then formed to define the Federal strategic priorities for AI R&D, with particular attention on areas that industry is unlikely to address. This National Artificial Intelligence R&D Strategic Plan establishes a set of objectives for Federally-funded AI research, both research occurring within the government as well as Federally-funded research occurring outside of government, such as in academia. The ultimate goal of this research is to produce new AI knowledge and technologies that provide a range of positive benefits to society, while minimizing the negative impacts.




Implementing the Government Performance and Results Act for Research


Book Description

As requested by Congress and the White House Office of Science and Technology Policy (OSTP), this report assists federal agencies in crafting plans and reports that are responsive to the Government Performance and Results Act (GPRA), OMB Guidance, and agency missions. Using a case study approach, the report identifies best practices used by individual agencies to evaluate the performance and results of their science and technology programs. The report takes into account individual agencies' missions and how science and technology programs and human resource needs are factored into agency GPRA plans. Specific applications of recommendations are included from COSEPUP's earlier report entitled Evaluating Federal Research Programs: Research and the Government Performance and Results Act.




The DARPA Model for Transformative Technologies: Perspectives on the U.S. Defense Advanced Research Projects Agency


Book Description

The authors have done a masterful job of charting the important story of DARPA, one of the key catalysts of technological innovation in US recent history. By plotting the development, achievements and structure of the leading world agency of this kind, this book stimulates new thinking in the field of technological innovation with bearing on how to respond to climate change, pandemics, cyber security and other global problems of our time. The DARPA Model provides a useful guide for governmental agency and policy leaders, and for anybody interested in the role of governments in technological innovation. —Dr. Kent Hughes, Woodrow Wilson International Center for Scholars This volume contains a remarkable collection of extremely insightful articles on the world’s most successful advanced technology agency. Drafted by the leading US experts on DARPA, it provides a variety of perspectives that in turn benefit from being presented together in a comprehensive volume. It reviews DARPA’s unique role in the U.S. innovation system, as well as the challenges DARPA and its clones face today. As the American model is being considered for adoption by a number of countries worldwide, this book makes a welcome and timely contribution to the policy dialogue on the role played by governments in stimulating technological innovation. — Prof. Charles Wessner, Georgetown University The U.S. Defense Advanced Research Projects Agency (DARPA) has played a remarkable role in the creation new transformative technologies, revolutionizing defense with drones and precision-guided munitions, and transforming civilian life with portable GPS receivers, voice-recognition software, self-driving cars, unmanned aerial vehicles, and, most famously, the ARPANET and its successor, the Internet. Other parts of the U.S. Government and some foreign governments have tried to apply the ‘DARPA model’ to help develop valuable new technologies. But how and why has DARPA succeeded? Which features of its operation and environment contribute to this success? And what lessons does its experience offer for other U.S. agencies and other governments that want to develop and demonstrate their own ‘transformative technologies’? This book is a remarkable collection of leading academic research on DARPA from a wide range of perspectives, combining to chart an important story from the Agency’s founding in the wake of Sputnik, to the current attempts to adapt it to use by other federal agencies. Informative and insightful, this guide is essential reading for political and policy leaders, as well as researchers and students interested in understanding the success of this agency and the lessons it offers to others.




Program Evaluation


Book Description

"To improve federal government performance and accountability, GPRAMA aims to ensure that agencies use performance information in decision making and holds them accountable for achieving results. The Office of Management and Budget (OMB) has encouraged agencies to strengthen their program evaluations- systematic studies of program performance-and expand their use in management and policy making. This report is one of a series in which GAO, as required by GPRAMA, examines the act's implementation. GAO examined federal agencies' capacity to conduct and use program evaluations and the activities and resources, including some related to GPRAMA, agencies found useful for building that capacity.GAO reviewed the literature to identify the key components and measures of evaluation capacity. GAO surveyed the PIOs of the 24 federal agencies subject to the Chief Financial Officers Act regarding their organizations' characteristics, expertise, and policies, and their observations on the usefulness of various resources and activities for building evaluation capacity. All 24 responded. GAO also interviewed OMB and Office of Personnel Management (OPM) staff about their capacity-building efforts."