Continuing the Journey to Reposition Culture and Cultural Context in Evaluation Theory and Practice


Book Description

Racial, ethnic, linguistic, and cultural diversity has become of global importance in places where many never would have imagined. Increasing diversity in the U.S., Europe, Africa, New Zealand, and Asia strongly suggests that a homogeneity-based focus is rapidly becoming an historical artifact. Therefore, culturally responsive evaluation (CRE) should no longer be viewed as a luxury or an option in our work as evaluators. The continued amplification of racial, ethnic, linguistic, and cultural diversity and awareness among the populations of the U.S. and other western nations insists that social science researchers and evaluators inextricably engage culturally responsive approaches in their work. It is unacceptable for most mainstream university evaluation programs, philanthropic agencies, training institutes sponsored by federal agencies, professional associations, and other entities to promote professional evaluation practices that do not attend to CRE. Our global demographics are a reality that can be appropriately described and studied within the context of complexity theory and theory of change (e.g., Stewart, 1991; Battram, 1999). And this perspective requires a distinct shift from “simple” linear cause-effect models and reductionist thinking to include more holistic and culturally responsive approaches. The development of policy that is meaningfully responsive to the needs of traditionally disenfranchised stakeholders and that also optimizes the use of limited resources (human, natural, and financial) is an extremely complex process. Fortunately, we are presently witnessing developments in methods, instruments, and statistical techniques that are mixed methods in their paradigm/designs and likely to be more effective in informing policymaking and decision-making. Culturally responsive evaluation is one such phenomenon that positions itself to be relevant in the context of dynamic international and national settings where policy and program decisions take place. One example of a response to address this dynamic and need is the newly established Center for Culturally Responsive Evaluation and Assessment (CREA) in the College of Education at the University of Illinois at Urbana-Champaign. CREA is an outgrowth of the collective work and commitments of a global community of scholars and practitioners who have contributed chapters to this edited volume. It is an international and interdisciplinary evaluation center that is grounded in the need for designing and conducting evaluations and assessments that embody cognitive, cultural, and interdisciplinary diversity so as to be actively responsive to culturally diverse communities and their aspirations. The Center’s purpose is to address questions, issues, theories, and practices related to CRE and culturally responsive educational assessment. Therefore, CREA can serve as a vehicle for our continuing discourse on culture and cultural context in evaluation and also as a point of dissemination for not only the work that is included in this edited volume, but for the subsequent work it will encourage.




The Future of Evaluation in Society


Book Description

The impetus for this volume lives in a rich and vibrant past. It is organized to honor one of the founders and most prolific contributors to the profession and transdiscipline of evaluation -- Professor Michael Scriven, and to illuminate the future of evaluation in society. Professor Scriven often shares stories of his meetings with Albert Einsten and the frame-breaking evaluation revolution he has led against the value free doctrine of the social sciences. Both his wide eyed graduate students and the more grizzled evaluation veterans in his professional development workshops quickly learn that Scriven is well traveled and has exchanged some of the boldest ideas and visions with the most brilliant thinkers of his time. Scriven insisted that the 2011 Stauffer Symposium and this volume be organized in that genre. He urged us to invite the most thoughtful and influential evaluation theorists and practitioners we could find to join him in a conversation about the future of evaluation in society. Scriven challenges us to examine the five great paradigm shifts that have revolutionized the foundations of evaluation, and that he believes will form the basis for a much brighter future for evaluation in society. Scriven’s revolutionary ideas are followed and challenged by a group of thought leaders in evaluation who do not necessarily shared his views on evaluation, but who have earned his deepest respect and whose evaluation work he admires including Michael Quinn Patton, Ernest House, Daniel Stufflebeam, Robert Stake, Jennifer Greene, Karen Kirkhart, Melvin Mark, Rodney Hopson, and Christina Christie. However, despite his insistence that his colleagues stay focused on the future of evaluation, you will find that many have recounted their adventures, exchanges, and debates with him over the years, as well as pointed out the many contributions that he has made to the development of evaluation and to the improvement of society through his amazing portfolio of evaluation contributions. The Future of Evaluation in Society: A Tribute to Michael Scriven will be of great interest to evaluation scholars, practitioners, and students of evaluation. It will be appropriate for use in a wide range of evaluation courses including Introduction to Evaluation, Evaluation Theory, and Evaluation Practice courses.




Fostering Computational Thinking Among Underrepresented Students in STEM


Book Description

This book broadly educates preservice teachers and scholars about current research on computational thinking (CT). More specifically, attention is given to computational algorithmic thinking (CAT), particularly among underrepresented K–12 student groups in STEM education. Computational algorithmic thinking (CAT)—a precursor to CT—is explored in this text as the ability to design, implement, and evaluate the application of algorithms to solve a variety of problems. Drawing on observations from research studies that focused on innovative STEM programs, including underrepresented students in rural, suburban, and urban contexts, the authors reflect on project-based learning experiences, pedagogy, and evaluation that are conducive to developing advanced computational thinking, specifically among diverse student populations. This practical text includes vignettes and visual examples to illustrate how coding, computer modeling, robotics, and drones may be used to promote CT and CAT among students in diverse classrooms.




Facilitating Evaluation


Book Description

Michael Quinn Patton’s Facilitating Evaluation: Principles in Practice is the first book of its kind to explain in depth and detail how to facilitate evaluation processes with stakeholders. Using the author’s own stories of his experiences as an evaluation facilitator, the book illustrates the five evaluation facilitation principles that are the organizing framework for addressing how to work with stakeholders to generate evaluation questions, make decisions among methods, interpret findings, and participate in all aspects of evaluation. Ultimately, this book will help readers perform facilitation to enhance the relevance, credibility, meaningfulness, and utility of evaluations. "A must-read for anyone considering a high-impact evaluation!" –Margaret Lombe, Boston College




Disrupting Program Evaluation and Mixed Methods Research for a More Just Society


Book Description

While women have influenced the fields of program evaluation and mixed methods research, their contributions are rarely acknowledged. In this edited volume, we recognize the contributions Jennifer C. Greene made to evaluation and mixed methods, focusing on how she disrupted traditional ways methods are conceptualized and implemented. We aim to provide a range of perspectives on how Greene disrupted social science theory, methodology, and training through her commitment to the values of democracy, diversity, and dialogue. Accordingly, the volume makes explicit how Greene democratized social science inquiry and training by emphasizing inclusive, culturally responsive, and educative theories, methods, and pedagogical practices. Second, it clarifies how her commitment to diversity supported scholars from diverse disciplinary backgrounds and cultures (within and beyond the United States) and validated often ignored perspectives on phenomena or experiences. And third, it showcases how her commitment to dialogue opened spaces for non-traditional methodological approaches that engage, communicate, and disseminate program evaluation and mixed methods work. Taken together, the first-hand accounts presented in this volume offer a historical perspective and practical guidance on program evaluation and mixed methods conceptualization, implementation, and training. Endorsements: 'The theme of “disruption” in this book is a powerful look at the breadth and significance of Jennifer C. Greene’s distinguished contributions, both theoretical and practical, in evaluation for over 30 years. In essence, these chapters reveal the sustaining power of her scholarship and influence today and likely in years to come. Her recognition that no single method can solve the complex problems faced in evaluation is revealed in the ontological, epistemological, and methodological work she developed on mixed methods, which led to a genre of study that recast the debate about the hegemony of quantitative over qualitative methods and vice versa. She has foregrounded values, the quintessential character of evaluation, by recognizing the multiplicity of perspectives that guide our theories, roles, questions, designs, and perceptions of quality. Jennifer Greene’s generativity is exemplified in her egalitarian dialogic stance, inclusion of voices and context, and commitment to equity. The chapters reflect the impact of her work on scholars who continue to advance evaluation for a more just society.' — Valerie J. Caracelli, U.S. Government Accountability Office 'It is not possible to convey in a few words my strongest endorsement for this volume that begins scratching the surface of Jennifer C. Greene’s many contributions to evaluation theory, methods, and practice. I have worked with and learned from her on many occasions over the years as I grappled with refining my understanding and practice of culturally responsive evaluation. I commend the editors and authors for this first substantive collection providing a more in-depth understanding and appreciation of Jennifer Greene’s body of work for some while providing others the opportunity to revisit the lessons we have learned from her work, spirit, and commitment to make a difference.' — Stafford Hood, Center for Culturally Responsive Evaluation & Assessment (CREA) and Professor Emeritus University of Illinois at Urbana-Champaign




Nonprofit Management 101


Book Description

A new edition of the essential guide to nonprofit management This intensely practical, comprehensive guidebook is for both leaders new to the nonprofit sector looking for a quick primer on all the issues that matter, as well as established veterans looking to understand how all the pieces fit together. Showcasing practical tips and takeaways, this how-to manual and resource guide provides easy to implement solutions for organizations seeking to expand impact and meet mission. Seasoned veterans including Van Jones, Fair Trade founder Paul Rice, Lynne Twist, Kay Sprinkel Grace, Joan Garry, and more share knowledge and useful insights on all aspects of nonprofit management, including: Fundraising from individuals, companies, and foundations Online fundraising, social networking, and effective use of technology Marketing, public relations, and events Board and volunteer engagement Human resources and career planning Lobbying and advocacy Legal and financial management Leadership and strategic planning This is essential reading for anyone in the nonprofit sector looking for the latest information in the field.




Qualitative Inquiry in Evaluation


Book Description

Integrate qualitative inquiry approaches and methods into the practice of evaluation Qualitative inquiry can have a major effect on evaluation practice, and provides evaluators a means to explore and examine various settings and contexts in need of rich description and deeper understanding. Qualitative Inquiry in Evaluation: From Theory to Practice explores the most important considerations for both students and evaluation professionals. Using various evaluation theories and approaches as a springboard for real-world practice, this reference serves as an accessible text for beginning students and seasoned professionals alike. Readers are given an in-depth view of the key qualities and benefits of qualitative inquiry, which also serves as a crucial counterpart to quantitative analysis. Chapters in part one focus on the foundations, core concepts, and intersection of evaluation theory and qualitative inquiry. Part two contains contributions from leading evaluators whose design, implementation, and reporting strategies for qualitative inquiry are centered on common, real-world settings. These case-based chapters point to the strengths and challenges of implementing qualitative evaluations. Key competencies for conducting effective qualitative evaluations are also discussed. Explores the role of qualitative inquiry in many prominent approaches to evaluation Discusses the method's history and delves into key concepts in qualitative inquiry and evaluation Helps readers understand which qualities are necessary to be an effective qualitative evaluator Presents the viewpoints and experiences of expert editors and contributing authors with high levels of understanding on the topic Qualitative Inquiry in Evaluation: From Theory to Practice is a vital tool for evaluators and students alike who are looking to deepen their understanding of the theoretical perspectives and practice considerations of qualitative evaluation.




Credible and Actionable Evidence


Book Description

Addressing one of the most important and contentious issues challenging applied research and evaluation practice today—what constitutes credible and actionable evidence?—this volume offers a balanced and current context in which to analyze the long-debated quantitative-qualitative paradigms. In the Second Edition, the contributors, a veritable "who’s who" in evaluation, discuss the diversity and changing nature of credible and actionable evidence; offer authoritative guidance about using credible and actionable evidence; explain how to use it to provide rigorous and influential evaluations; and include lessons from their own applied research and evaluation to suggest ways to address the key issues and challenges. Reflecting the latest developments in the field and covering both experimental and non-experimental methods, the new edition includes revised and updated chapters, summaries of strengths and weaknesses across varied approaches, and contains diverse definitions of evidence. Also included are two new chapters on assessing credibility and synthesizing evidence for policy makers. This is a valuable resource for students and others interested in how to best study and evaluate programs, policies, organizations, and other initiatives designed to improve aspects of the human condition and societal well-being.




Handbook of Practical Program Evaluation


Book Description

The leading program evaluation reference, updated with the latest tools and techniques The Handbook of Practical Program Evaluation provides tools for managers and evaluators to address questions about the performance of public and nonprofit programs. Neatly integrating authoritative, high-level information with practicality and readability, this guide gives you the tools and processes you need to analyze your program's operations and outcomes more accurately. This new fourth edition has been thoroughly updated and revised, with new coverage of the latest evaluation methods, including: Culturally responsive evaluation Adopting designs and tools to evaluate multi-service community change programs Using role playing to collect data Using cognitive interviewing to pre-test surveys Coding qualitative data You'll discover robust analysis methods that produce a more accurate picture of program results, and learn how to trace causality back to the source to see how much of the outcome can be directly attributed to the program. Written by award-winning experts at the top of the field, this book also contains contributions from the leading evaluation authorities among academics and practitioners to provide the most comprehensive, up-to-date reference on the topic. Valid and reliable data constitute the bedrock of accurate analysis, and since funding relies more heavily on program analysis than ever before, you cannot afford to rely on weak or outdated methods. This book gives you expert insight and leading edge tools that help you paint a more accurate picture of your program's processes and results, including: Obtaining valid, reliable, and credible performance data Engaging and working with stakeholders to design valuable evaluations and performance monitoring systems Assessing program outcomes and tracing desired outcomes to program activities Providing robust analyses of both quantitative and qualitative data Governmental bodies, foundations, individual donors, and other funding bodies are increasingly demanding information on the use of program funds and program results. The Handbook of Practical Program Evaluation shows you how to collect and present valid and reliable data about programs.




Focus Groups


Book Description

A 2021 AERA Qualitative Research SIG Outstanding Book Award winner Focus Groups: Culturally Responsive Approaches for Qualitative Inquiry and Program Evaluation takes an in-depth look at how culturally-responsive focus groups are developed and implemented within the context of qualitative inquiry broadly, and program evaluation more specifically. The book showcases various forms of focus groups and how they can be responsive to specific communities across different disciplines. This book provides: an historical perspective on focus groups a theoretical foundation helpful for supporting focus groups with marginalized groups, vulnerable populations (older adults, children), and participants within non-Western settings basic procedures for conducting focus groups guidelines for cultural responsiveness and case examples of alternative approaches to focus groups that target specific communities and those in different contexts. It also considers ethical issues around the use of culturally responsive focus groups, while providing guidance on analyzing and interpreting your data and establishing the credibility of your study. Focus Groups is an information rich resource for qualitative researchers and program evaluators at various levels who want to learn about or enhance their knowledge on designing and conducting culturally responsive focus groups. Perfect for courses such as: Foundations of Culturally Responsive Focus Groups | Culturally Responsive Approaches to Program Evaluation Practicing Qualitative Program Evaluation | Qualitative Research: Culturally Responsive Methods | Focus Groups in Research | Introduction to Qualitative Methods | Methods in Educational Research | Program Evaluation Practice | Designing Qualitative Research