Applying Generalizability Theory using EduG


Book Description

Intended to help improve measurement and data collection methods in the behavioral, social, and medical sciences, this book demonstrates an expanded and accessible use of Generalizability Theory (G theory). G theory conceptually models the way in which the reliability of measurement is ascertained. Sources of score variation are identified as potential contributors to measurement error and taken into account accordingly. The authors demonstrate the powerful potential of G theory by showing how to improve the quality of any kind of measurement, regardless of the discipline. Readers will appreciate the conversational style used to present a comprehensive review of G theory and its application using the freeware EduG. To maximize understanding the authors illustrate all fundamental principles with concrete examples from different fields and contexts. Annotated applications lead students through the main concepts of G theory, while illustrating both the use of EduG and interpretation of its output. Formulas are avoided wherever possible. Exercises with data sets available on the Psychology Press website allow readers to carry out their own analyses to reinforce understanding. Brief overviews of analysis of variance, estimation, and the statistical error model are provided for review. The procedures involved in carrying out a generalizability study using EduG follow, as well as guidance in the interpretation of results. Real-world applications of G theory to the assessment of depression, managerial ability, attitudes, and writing and mathematical skills are then presented. Next, annotated exercises provide an opportunity for readers to use EduG and interpret its results. The book concludes with a review of the development of G theory and possible new directions of application. Finally, for those with a strong statistical background, the appendixes provide the formulas used by EduG. Ideal as a supplement for courses on measurement theory and/or generalizability theory taught in departments of psychology, education, medicine, and the social sciences, this text will also appeal to researchers from a variety of fields interested in learning how to apply G theory to their studies.




Generalizability Theory


Book Description

Generalizability theory offers an extensive conceptual framework and a powerful set of statistical procedures for characterizing and quantifying the fallibility of measurements. Robert Brennan, the author, has written the most comprehensive and up-to-date treatment of generalizability theory. The book provides a synthesis of those parts of the statistical literature that are directly applicable to generalizability theory. The principal intended audience is measurement practitioners and graduate students in the behavioral and social sciences, although a few examples and references are provided from other fields. Readers will benefit from some familiarity with classical test theory and analysis of variance, but the treatment of most topics does not presume specific background.







Generalizability Theory


Book Description

Accessible to any professional or researcher who has a basic understanding of analysis of variance, Shavelson and Webb offer an intuitive development of generalizability theory, a technique for estimating the relative magnitudes of various components of error variation and for indicating the most efficient strategy for achieving desired measurement precision. Covering a variety of topics such as generalizability studies with nested facets and with fixed facets, measurement error and generalizability coefficients, and decision studies with same and with different designs, the text includes exercises so the reader may practice the application of each chapter's material. By using detailed illustrations and examples, Shavelson and Webb clearly describe the logic underlying major concepts in generalizability theory to enable readers to apply these methods when investigating the consistency of their own measurements.




Handbook on Measurement, Assessment, and Evaluation in Higher Education


Book Description

In this valuable resource, well-known scholars present a detailed understanding of contemporary theories and practices in the fields of measurement, assessment, and evaluation, with guidance on how to apply these ideas for the benefit of students and institutions. Bringing together terminology, analytical perspectives, and methodological advances, this second edition facilitates informed decision-making while connecting the latest thinking in these methodological areas with actual practice in higher education. This research handbook provides higher education administrators, student affairs personnel, institutional researchers, and faculty with an integrated volume of theory, method, and application.




The SAGE Encyclopedia of Research Design


Book Description

The SAGE Encyclopedia of Research Design maps out how one makes decisions about research design, interprets data, and draws valid inferences, undertakes research projects in an ethical manner, and evaluates experimental design strategies and results. From A-to-Z, this four-volume work covers the spectrum of research design strategies and topics including, among other things: fundamental research design principles, ethics in the research process, quantitative versus qualitative and mixed-method designs, completely randomized designs, multiple comparison tests, diagnosing agreement between data and models, fundamental assumptions in analysis of variance, factorial treatment designs, complete and incomplete block designs, Latin square and related designs, hierarchical designs, response surface designs, split-plot designs, repeated measures designs, crossover designs, analysis of covariance, statistical software packages, and much more. Research design, with its statistical underpinnings, can be especially daunting for students and novice researchers. At its heart, research design might be described simply as a formalized approach toward problem solving, thinking, and acquiring knowledge, the success of which depends upon clearly defined objectives and appropriate choice of statistical design and analysis to meet those objectives. The SAGE Encyclopedia of Research Design will assist students and researchers with their work while providing vital information on research strategies.




Educational and Psychological Measurement


Book Description

This new text provides the most current coverage of measurement and psychometrics in a single volume. Authors W. Holmes Finch and Brian F. French first review the basics of psychometrics and measurement, before moving on to more complex topics such as equating and scaling, item response theory, standard setting, and computer adaptive testing. Also included are discussions of cutting-edge topics utilized by practitioners in the field, such as automated test development, game-based assessment, and automated test scoring. This book is ideal for use as a primary text for graduate-level psychometrics/measurement courses, as well as for researchers in need of a broad resource for understanding test theory. Features: "How it Works" and "Psychometrics in the Real World" boxes break down important concepts through worked examples, and show how theory can be applied to practice. End-of-chapter exercises allow students to test their comprehension of the material, while suggested readings and website links provide resources for further investigation. A collection of free online resources include the full output from R, SPSS, and Excel for each of the analyses conducted in the book, as well as additional exercises, sample homework assignments, answer keys, and PowerPoint lecture slides.




Methodological Advances in Educational Effectiveness Research


Book Description

Methodological Advances in Educational Effectiveness Research is an important new work by some of the leading researchers in the field of Educational Effectiveness Research (EER). The book provides a state of the art snapshot of the methodology of EER now and clearly demonstrates the way it is applied in both research and evaluation. It shows how developments in the research methodology area such as the use of multilevel modelling approaches to analyse nested data have promoted the knowledge-base of educational effectiveness. But at the same time, as the authors show, the knowledge-base of educational effectiveness and the attempt to establish theoretical models do paradoxically challenge the development of methodologically appropriate studies including ways of analysing data. Guiding readers though the effective and appropriate use in educational effectiveness of: Longitudinal Studies Experimental Studies Mixed Research Methods Meta-analyses of effectiveness studies Using IRT to measure outcomes and factors Using Generalisability Theory to test the quality of data Multilevel modelling , and Structural Equation Modelling Techniques The authors draw in the expertise of scholars from around the world to show the mathematical background of each technique, the current and future applications, and Specific examples of applying this orientation to help the readers design their own effectiveness studies using specific methodological tools.




Quantitative Data Analysis for Language Assessment Volume I


Book Description

Quantitative Data Analysis for Language Assessment Volume I: Fundamental Techniques is a resource book that presents the most fundamental techniques of quantitative data analysis in the field of language assessment. Each chapter provides an accessible explanation of the selected technique, a review of language assessment studies that have used the technique, and finally, an example of an authentic study that uses the technique. Readers also get a taste of how to apply each technique through the help of supplementary online resources that include sample data sets and guided instructions. Language assessment students, test designers, and researchers should find this a unique reference as it consolidates theory and application of quantitative data analysis in language assessment.




Educational Sciences I


Book Description