Assessing Research: The Researchers' View, Volume 2


Book Description

This report, prepared for and funded by the Joint Funding Bodies' Review of Research Assessment, presents findings from a series of nine facilitated workshops held with academics and research managers across the United Kingdom (UK) in December 2002. The objective of the workshops was to investigate views of research quality and attitudes towards different models of research assessment. The report outlines the recurring themes and issues raised by the 142 participants in the workshops. The participants, academics and research managers, represented over one third of the 173 institutions that submitted to the Research Assessment Exercise in 2001. This report will be of interest to those concerned with research assessment and evaluation in academic research, both practitioners and policy makers. In the first workshop task, participants considered the characteristics of high quality research and how it should be assessed. In the second task, participants discussed the strengths and weaknesses of four approaches to research assessment: Expert Review, Algorithms, Historical Ratings, and Self Assessment. In the remaining two tasks, participants were asked to design their ideal assessment system, basing it on one of the approaches examined in Task 2. They then considered how their system would be implemented, what its weak points might be, and how its use would change research culture in UK higher education. The overwhelming majority of the workshop participants felt that research should be assessed using a system based on peer review by subject-based panels. Of the 29 systems designed, 25 were based on Expert Review. The participants also indicated that these panels should be informed by metrics and self-assessment, with some input from research users. The first volume of this report describes the methodology and details the findings of the workshops. The second volume contains additional source data. (66 tables, 1 figure).




Assessing Research


Book Description

The Research Assessment Exercise is used to assess the quality of, and determine core government funding for, research carried out in UK universities. In 2002 this process was reviewed (http://www.ra-review.ac.uk). This report provided evidence for the review; it describes workshops that investigated the academic community's views of research quality and attitudes towards models of assessment. Nine facilitated workshops were held in December 2002, which involved 142 academics and research managers from throughout the UK. The report outlines the recurring themes and issues raised by the participants. Key findings were that the overwhelming majority of participants thought research should be assessed using a system based on peer review; that there was a strong desire for a system with clear rules and transparent procedures; and that there was a need for improved systems to assess inter- and multi-disciplinary research work.




Assessing Research: The Researchers' View


Book Description

This report, prepared for and funded by the Joint Funding Bodies' Review of Research Assessment, presents findings from a series of nine facilitated workshops held with academics and research managers across the United Kingdom (UK) in December 2002. The objective of the workshops was to investigate views of research quality and attitudes towards different models of research assessment. The report outlines the recurring themes and issues raised by the 142 participants in the workshops. The participants, academics and research managers, represented over one third of the 173 institutions that submitted to the Research Assessment Exercise in 2001. This report will be of interest to those concerned with research assessment and evaluation in academic research, both practitioners and policy makers. In the first workshop task, participants considered the characteristics of high quality research and how it should be assessed. In the second task, participants discussed the strengths and weaknesses of four approaches to research assessment: Expert Review, Algorithms, Historical Ratings, and Self Assessment. In the remaining two tasks, participants were asked to design their ideal assessment system, basing it on one of the approaches examined in Task 2. They then considered how their system would be implemented, what its weak points might be, and how its use would change research culture in UK higher education. The overwhelming majority of the workshop participants felt that research should be assessed using a system based on peer review by subject-based panels. Of the 29 systems designed, 25 were based on Expert Review. The participants also indicated that these panels should be informed by metrics and self-assessment, with some input from research users. The first volume of this report describes the methodology and details the findings of the workshops. The second volume contains additional source data. (66 tables, 1 figure).




Assessing Research


Book Description

The Research Assessment Exercise is used to assess the quality of, and determine core government funding for, research carried out in UK universities. This report provided evidence for a review of the process.




Assessing Research


Book Description

The Research Assessment Exercise was reviewed in 2002. This seminar report concluded that research should be assessed using a system based on peer review; the system should have clear rules and transparent procedures; and that there was a need for improved systems to assess inter- and multi-disciplinary research work.




Finding What Works in Health Care


Book Description

Healthcare decision makers in search of reliable information that compares health interventions increasingly turn to systematic reviews for the best summary of the evidence. Systematic reviews identify, select, assess, and synthesize the findings of similar but separate studies, and can help clarify what is known and not known about the potential benefits and harms of drugs, devices, and other healthcare services. Systematic reviews can be helpful for clinicians who want to integrate research findings into their daily practices, for patients to make well-informed choices about their own care, for professional medical societies and other organizations that develop clinical practice guidelines. Too often systematic reviews are of uncertain or poor quality. There are no universally accepted standards for developing systematic reviews leading to variability in how conflicts of interest and biases are handled, how evidence is appraised, and the overall scientific rigor of the process. In Finding What Works in Health Care the Institute of Medicine (IOM) recommends 21 standards for developing high-quality systematic reviews of comparative effectiveness research. The standards address the entire systematic review process from the initial steps of formulating the topic and building the review team to producing a detailed final report that synthesizes what the evidence shows and where knowledge gaps remain. Finding What Works in Health Care also proposes a framework for improving the quality of the science underpinning systematic reviews. This book will serve as a vital resource for both sponsors and producers of systematic reviews of comparative effectiveness research.







The Critical Assessment of Research


Book Description

This book examines the following factors: sponsorship of research, control of the dissemination of research, effects of dominant research paradigms, financial interests of authors, publishers, and editors, role of new technologies (for example, Web 2.0).It is widely accepted among researchers and educators that the peer review process, the reputation of the publisher and examination of the author's credentials are the gold standards for assessing the quality of research and information. However, the traditional gold standards are not sufficient, and the effective evaluation of information requires the consideration of additional factors. Controversies about positive evaluations of new medications that appear in peer-reviewed journals, the financial reports on Enron prior to the revelations that led to its collapse, and obstacles to the publication of research that does not conform to dominant paradigms are just a few examples that indicate the need for a more sophisticated and nuanced approach to evaluating information.Each of the factors is discussed in a factual manner, supported by many examples that illustrate not only the nature of the issues but also their complexity. Practical suggestions for the evaluation of information are an integral part of the text. Highlights frequently overlooked criteria for evaluating research Challenges the assumption that the gold standards for evaluation are sufficient Examines the role of new technologies in evaluating and disseminating research




Handbook on Research Assessment in the Social Sciences


Book Description

This Handbook provides a comprehensive overview of current developments, issues and good practices regarding assessment in social science research. It pays particular attention to the challenges in evaluation policies in the social sciences, as well as to the specificities of publishing in the area. The Handbook discusses the current societal challenges facing researchers, from digital societies, to climate change and sustainability, to trust in democratic societies. Chapters provide ways to strengthen research assessment in the social sciences for the better, by offering a diverse range of experiences and views of experts from all continents. The Handbook also outlines major data sources that can be used to assess social sciences research, as well as looking at key dimensions of research quality in the social sciences including journal peer review, the issue of identifying research quality, and gender disparities in social science research. This book will be an essential read for scholars interested in research assessment in the social sciences. It will also be useful to policy makers looking to understand the key position of the social sciences in science and society and provide appropriate frameworks for key societal challenges.




Social Science Research


Book Description

This book is designed to introduce doctoral and graduate students to the process of conducting scientific research in the social sciences, business, education, public health, and related disciplines. It is a one-stop, comprehensive, and compact source for foundational concepts in behavioral research, and can serve as a stand-alone text or as a supplement to research readings in any doctoral seminar or research methods class. This book is currently used as a research text at universities on six continents and will shortly be available in nine different languages.