Understanding Measurement: Reliability
1st Edition
Published on 16. September 2010
Book
Paperback/Softback
160 pages
978-0-19-538036-1 (ISBN)
Description
This is a title in our Understanding Statistics series, which is designed to provide researchers with authoritative guides to understanding, presenting and critiquing analyses and associated inferences. Each volume in the series demonstrates how the relevant topic should be reported -- including detail surrounding what can be said, and how it should be said, as well as drawing boundaries around what cannot appropriately be claimed or inferred.
This volume addresses reliability, which is a fundamental aspect of any social science study that involves educational or psychological measurement. It not only has implications for the quality of test scores themselves,
but also any statistical analysis conducted using those scores. Topics addressed in this book include cover three different types of reliability methods and appropriate standard errors of measurement: classical test theory methods, decision consistency indices, and generalizability theory coeffcients. After a brief introduction to the topic, the author outlines how to report reliability in professional journal articles. Meyer is known for his clear, accessible writing; like all books in this series, this volume includes examples of both good and bad write-ups for methods sections of journal articles.
This volume addresses reliability, which is a fundamental aspect of any social science study that involves educational or psychological measurement. It not only has implications for the quality of test scores themselves,
but also any statistical analysis conducted using those scores. Topics addressed in this book include cover three different types of reliability methods and appropriate standard errors of measurement: classical test theory methods, decision consistency indices, and generalizability theory coeffcients. After a brief introduction to the topic, the author outlines how to report reliability in professional journal articles. Meyer is known for his clear, accessible writing; like all books in this series, this volume includes examples of both good and bad write-ups for methods sections of journal articles.
More details
Series
Language
English
Place of publication
New York
United States
Target group
Professional and scholarly
graduate students and scholars in psychology and other social science disciplines
Dimensions
Height: 210 mm
Width: 140 mm
Thickness: 9 mm
Weight
205 gr
ISBN-13
978-0-19-538036-1 (9780195380361)
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Other editions
Additional editions
Patrick Meyer
Understanding Measurement: Reliability
E-Book
04/2010
1st Edition
OUP eBook
€27.49
Available for download
Person
Patrick Meyer, Assistant Professor, Curry School of Education, University of Virginia.
Content
Chapter I: Introduction
The Importance of Test Score Reliability
Organization of the Book
General Concepts
The Measurement Procedure
Sampling in Measurement
Sampling the Measurement Procedure.
Sampling Examinees.
Replicating the Measurement Procedure and Sampling Examinees
Classical Test Theory
Replicating the Measurement Procedure
Additional Restrictions that Define Classical Test Theory
Sources of Variance
The Reliability Coefficient
Estimating Unobserved Quantities
Classification decisions
Replicating the Measurement Procedure
Strong True Score Theory
Distributional Assumptions
Domain Scores.
Conditional Distribution of Observed Scores.
Observed Scores.
Estimating Domain Scores.
Reliability and Conditional Standard Error of Measurement
Generalizability Theory
Generalizability Study
Variance Components.
Decision Study
Replicating the Measurement Procedure.
Sources of Variance.
Types of Decisions.
Reliability Coefficients.
Estimating Unobserved Quantities.
Chapter Summary
Chapter 2: Data Collection Designs
Description of Real-World Measurement Procedure
Data Collections Designs in Classical Test Theory
Whole Test Replications
Test-Retest Reliability Coefficient
Alternate Forms Reliability Coefficient
Test-Retest with Alternate Forms
Two Part-Test Replications
Split-halves
Other Two-Part Divisions.
Multiple Part-test Divisions.
Data Collection Designs for Classification Decisions
Data Collection Designs in Generalizability Theory
Common Data Collection Designs
Random Designs
Mixed Designs
Classical Test Theory Designs Restated as Data Collection Designs
Data Collection Designs for the ELA Benchmark Assessment
Chapter Summary
Chapter 3: Assumptions
Dimensionality
Violation of Unidimensionality
Error Score Correlations
Violation of Uncorrelated Errors
The Nature of Measurement Procedure Replications
Classical Test Theory
Congeneric Measures.
Classical Congeneric Measures.
Essentially Tau-equivalent Measures.
Tau-equivalent Measures.
Parallel Measures.
Violation of Essential tau-equivalence
Classification Decisions and Generalizability Theory
Randomly Parallel Measures
Distributional Assumptions
Violation of Randomly Parallel Measures
Chapter Summary
Chapter 4: Methods
Reliability Coefficients for Relative Decisions
Whole Test Replications
Two Part-test Replications
Multiple Part-test Replications
Other Methods for Estimating Reliability
Estimating Reliability for Absolute Decisions
Two Whole-test Replications
Squared Error Loss Coefficients
Threshold Loss Indices
One Whole-test Replication
Squared Error Loss Coefficients
Threshold Loss Indices
Other Methods
Chapter Summary
Chapter 5: Results
Documentation of Score Reliability
Characterize the Examinee Population
Describe the Measurement Procedure and All Major Sources of Error
Present Evidence in Support of Assumptions
Provide Estimates of Reliability and the Standard Error of Measurement
Benchmark ELA Assessment
Sample Results Section
Palmetto Achievement Challenge Test of Mathematics
Sample Results Section
The Responsive Classroom Efficacy Study and MSCAN
Sample Results Section
Chapter Summary
Chapter 6: Discussion and Recommended Readings
ELA Benchmark Assessment
Some Interesting Points about the Analysis
Palmetto Achievement Challenge Test of Mathematics
The Responsive Classroom Efficacy Study and MSCAN
Recommended Readings
References
The Importance of Test Score Reliability
Organization of the Book
General Concepts
The Measurement Procedure
Sampling in Measurement
Sampling the Measurement Procedure.
Sampling Examinees.
Replicating the Measurement Procedure and Sampling Examinees
Classical Test Theory
Replicating the Measurement Procedure
Additional Restrictions that Define Classical Test Theory
Sources of Variance
The Reliability Coefficient
Estimating Unobserved Quantities
Classification decisions
Replicating the Measurement Procedure
Strong True Score Theory
Distributional Assumptions
Domain Scores.
Conditional Distribution of Observed Scores.
Observed Scores.
Estimating Domain Scores.
Reliability and Conditional Standard Error of Measurement
Generalizability Theory
Generalizability Study
Variance Components.
Decision Study
Replicating the Measurement Procedure.
Sources of Variance.
Types of Decisions.
Reliability Coefficients.
Estimating Unobserved Quantities.
Chapter Summary
Chapter 2: Data Collection Designs
Description of Real-World Measurement Procedure
Data Collections Designs in Classical Test Theory
Whole Test Replications
Test-Retest Reliability Coefficient
Alternate Forms Reliability Coefficient
Test-Retest with Alternate Forms
Two Part-Test Replications
Split-halves
Other Two-Part Divisions.
Multiple Part-test Divisions.
Data Collection Designs for Classification Decisions
Data Collection Designs in Generalizability Theory
Common Data Collection Designs
Random Designs
Mixed Designs
Classical Test Theory Designs Restated as Data Collection Designs
Data Collection Designs for the ELA Benchmark Assessment
Chapter Summary
Chapter 3: Assumptions
Dimensionality
Violation of Unidimensionality
Error Score Correlations
Violation of Uncorrelated Errors
The Nature of Measurement Procedure Replications
Classical Test Theory
Congeneric Measures.
Classical Congeneric Measures.
Essentially Tau-equivalent Measures.
Tau-equivalent Measures.
Parallel Measures.
Violation of Essential tau-equivalence
Classification Decisions and Generalizability Theory
Randomly Parallel Measures
Distributional Assumptions
Violation of Randomly Parallel Measures
Chapter Summary
Chapter 4: Methods
Reliability Coefficients for Relative Decisions
Whole Test Replications
Two Part-test Replications
Multiple Part-test Replications
Other Methods for Estimating Reliability
Estimating Reliability for Absolute Decisions
Two Whole-test Replications
Squared Error Loss Coefficients
Threshold Loss Indices
One Whole-test Replication
Squared Error Loss Coefficients
Threshold Loss Indices
Other Methods
Chapter Summary
Chapter 5: Results
Documentation of Score Reliability
Characterize the Examinee Population
Describe the Measurement Procedure and All Major Sources of Error
Present Evidence in Support of Assumptions
Provide Estimates of Reliability and the Standard Error of Measurement
Benchmark ELA Assessment
Sample Results Section
Palmetto Achievement Challenge Test of Mathematics
Sample Results Section
The Responsive Classroom Efficacy Study and MSCAN
Sample Results Section
Chapter Summary
Chapter 6: Discussion and Recommended Readings
ELA Benchmark Assessment
Some Interesting Points about the Analysis
Palmetto Achievement Challenge Test of Mathematics
The Responsive Classroom Efficacy Study and MSCAN
Recommended Readings
References