Methods of Meta-Analysis
Correcting Error and Bias in Research Findings
2nd Edition
Published on 3. June 2004
Book
Paperback/Softback
616 pages
978-1-4129-0479-7 (ISBN)
Description
Click 'Additional Materials' for downloadable sample chapter
"Clearly written and compellingly argued, this book explains the procedures and benefits of correcting for measurement error and range restriction and details the methodological developments in meta-analysis over the last decade. No one should consider conducting a meta-analysis without first reading this book. It is essential reading for all scientists."
-Michael A. McDaniel, Virginia Commonwealth University
"A book that will certainly appeal not only to the students, but will also be a great reference source for the technically sophisticated professional. The breadth and depth of the coverage, not to mention the novelty and clarity of writing, makes this book a classic in the field. It covers (and at times introduces) many novel issues that will be in the forefront for some years to come-as such a must read for all meta-analysts."
-Vish C. Viswesvaran, Ph.D., Director of I/O Program, Florida International University
Meta-analysis is arguably the most important methodological innovation in the social and behavioral sciences in the last 25 years. Developed to offer researchers an informative account of which methods are most useful in integrating research findings across studies, this book will enable the reader to apply, as well as understand, meta-analytic methods. Rather than taking an encyclopedic approach, the authors have focused on carefully developing those techniques that are most applicable to social science research, and have given a general conceptual description of more complex and rarely-used techniques. Fully revised and updated, Methods of Meta-Analysis, Second Edition is the most comprehensive text on meta-analysis available today.
New to the Second Edition:
* An evaluation of fixed versus random effects models for meta-analysis
* New methods for correcting for indirect range restriction in meta-analysis
* New developments in corrections for measurement error
* A discussion of a new Windows-based program package for applying the meta-analysis methods presented in the book
* A presentation of the theories of data underlying different approaches to meta-analysis
"Clearly written and compellingly argued, this book explains the procedures and benefits of correcting for measurement error and range restriction and details the methodological developments in meta-analysis over the last decade. No one should consider conducting a meta-analysis without first reading this book. It is essential reading for all scientists."
-Michael A. McDaniel, Virginia Commonwealth University
"A book that will certainly appeal not only to the students, but will also be a great reference source for the technically sophisticated professional. The breadth and depth of the coverage, not to mention the novelty and clarity of writing, makes this book a classic in the field. It covers (and at times introduces) many novel issues that will be in the forefront for some years to come-as such a must read for all meta-analysts."
-Vish C. Viswesvaran, Ph.D., Director of I/O Program, Florida International University
Meta-analysis is arguably the most important methodological innovation in the social and behavioral sciences in the last 25 years. Developed to offer researchers an informative account of which methods are most useful in integrating research findings across studies, this book will enable the reader to apply, as well as understand, meta-analytic methods. Rather than taking an encyclopedic approach, the authors have focused on carefully developing those techniques that are most applicable to social science research, and have given a general conceptual description of more complex and rarely-used techniques. Fully revised and updated, Methods of Meta-Analysis, Second Edition is the most comprehensive text on meta-analysis available today.
New to the Second Edition:
* An evaluation of fixed versus random effects models for meta-analysis
* New methods for correcting for indirect range restriction in meta-analysis
* New developments in corrections for measurement error
* A discussion of a new Windows-based program package for applying the meta-analysis methods presented in the book
* A presentation of the theories of data underlying different approaches to meta-analysis
Reviews / Votes
"Clearly written and compellingly argued, this book explains the procedures and benefits of correcting for measurement error and range restriction and details the methodological developments in meta-analysis over the last decade. No one should consider conducting a meta-analysis without first reading this book. It is essential reading for all scientists." -- Michael A. McDaniel "A book that will certainly appeal not only to the students, but will also be a great reference source for the technically sophisticated professional. The breadth and depth of the coverage, not to mention the novelty and clarity of writing, makes this book a classic in the field. It covers (and at times introduces) many novel issues that will be in the forefront for some years to come-as such a must read for all meta-analysts." -- Vish C. Viswesvaran, Ph.D.More details
Edition
2nd Revised edition
Language
English
Place of publication
Thousand Oaks
United States
Target group
Professional and scholarly
College/higher education
Edition type
Revised edition
Dimensions
Height: 254 mm
Width: 178 mm
Thickness: 33 mm
Weight
1145 gr
ISBN-13
978-1-4129-0479-7 (9781412904797)
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Other editions
Previous edition
John E. Hunter | Frank L. Schmidt
Methods of Meta-Analysis
Correcting Error and Bias in Research Findings
Book
02/1990
1st Edition
SAGE Publications Inc
€84.38
Article exhausted; check for reprint
Persons
John E. (Jack) Hunter (1939--2002) was a professor in the Department of Psychology at Michigan State University. He received his Ph.D. in quantitative psychology from the University of Illinois. Jack coauthored four books and authored or coauthored over 200 articles and book chapters on a wide variety of methodological topics, including confirmatory and exploratory factor analysis, measurement theory and methods, statistics, and research methods. He also published numerous research articles on such substantive topics as intelligence, attitude change, the relationship between attitudes and behavior, validity generalization, differential validity/selection fairness, and selection utility. Much of his research on attitudes was in the field of communications, and the American Communications Association named a research award in his honor. Professor Hunter received the Distinguished Scientific Award for Contributions to Applied Psychology from the American Psychological Association (APA) (jointly with Frank Schmidt) and the Distinguished Scientific Contributions Award from the Society for Industrial/Organizational Psychology (SIOP) (also jointly with Frank Schmidt). He was a Fellow of APA, APS, and SIOP, and was a past president of the Midwestern Society for Multivariate Experimental Psychology. For the story of Jack's life, see Schmidt (2003).
Frank L. Schmidt is the Gary F. Fethke Leadership Professor Emeritus in the Department of Management and Organization in the Tippie College of Business at the University of Iowa. He received his Ph.D. in industrial/organizational psychology from Purdue University and has been on the faculties of Michigan State and George Washington Universities. He has authored or coauthored seven books and nearly 200 articles and book chapters on measurement, statistics, research methods, individual differences, and personnel selection. He headed a research program in the U.S. Office of Personnel Management in Washington, D.C., for 11 years, during which time he published numerous research studies in personnel psychology, primarily with John Hunter. Their research on the generalizability of employment selection method validities led to the development of the meta-analysis methods presented in this book. Professor Schmidt has received the Distinguished Scientific Award for Contributions to Applied Psychology from the American Psychological Association (APA) (jointly with John Hunter) and the Distinguished Scientific Contributions Award from the Society for Industrial/Organizational Psychology (SIOP) (also jointly with John Hunter). He has also received the Ingram Olkin Award and the Frederick Mosteller Award, both for contributions to meta-analysis methodology; the Scientific Award for Applications of Psychology from the Association for Psychological Science (APS); the Gold Medal Lifetime Achievement award from the APA Foundation; the Distinguished Career Award for Contributions to Human Resources, and the Distinguished Career Achievement Award for Contributions to Research Methods, both from the Academy of Management. He is a Fellow of the APA, the Association for Psychological Science, and SIOP, and is past president of Division 5 (Measurement, Statistics, & Evaluation) of the APA.
Frank L. Schmidt is the Gary F. Fethke Leadership Professor Emeritus in the Department of Management and Organization in the Tippie College of Business at the University of Iowa. He received his Ph.D. in industrial/organizational psychology from Purdue University and has been on the faculties of Michigan State and George Washington Universities. He has authored or coauthored seven books and nearly 200 articles and book chapters on measurement, statistics, research methods, individual differences, and personnel selection. He headed a research program in the U.S. Office of Personnel Management in Washington, D.C., for 11 years, during which time he published numerous research studies in personnel psychology, primarily with John Hunter. Their research on the generalizability of employment selection method validities led to the development of the meta-analysis methods presented in this book. Professor Schmidt has received the Distinguished Scientific Award for Contributions to Applied Psychology from the American Psychological Association (APA) (jointly with John Hunter) and the Distinguished Scientific Contributions Award from the Society for Industrial/Organizational Psychology (SIOP) (also jointly with John Hunter). He has also received the Ingram Olkin Award and the Frederick Mosteller Award, both for contributions to meta-analysis methodology; the Scientific Award for Applications of Psychology from the Association for Psychological Science (APS); the Gold Medal Lifetime Achievement award from the APA Foundation; the Distinguished Career Award for Contributions to Human Resources, and the Distinguished Career Achievement Award for Contributions to Research Methods, both from the Academy of Management. He is a Fellow of the APA, the Association for Psychological Science, and SIOP, and is past president of Division 5 (Measurement, Statistics, & Evaluation) of the APA.
Content
Preface to 2nd Edition
Preface to 1st Edition
Acknowledgements
Introduction to Meta-Analysis
Integration Research Findings Across Studies
General problem and an example
Problems with statistical significance tests
Is statistical power the solution?
Confidence intervals
Meta-analysis
Role of meta-analysis in the behavioral and social sciences
Role of meta-analysis in theory development
Increasing use of meta-analysis
Meta-analysis in industrial-organizational psychology
Wider impact of meta-analysis on psychology
Impact of meta-analysis outside psychology
Meta-analysis and social policy
Meta-analysis and theories of data
Conclusions
Study Artifacts and Their Impact on Study Outcomes
Study Artifacts
Sampling error, statistical power, and the interpretation of research literatures
When and how to cumulate
Undercorrection for artifacts in the corrected standard deviation
Coding study characteristics and capitalization on sampling error in moderator analysis
A look ahead in the book
Meta-Analysis of Correlations
Meta-Analysis of Correlations Corrected Individually for Artifacts
Introduction and Overview
Bare bones meta-analysis: Correcting for sampling error only
Artifacts other than sampling error
Multiple simultaneous artifacts
Meta-analysis of individually corrected correlations
A worked example: Indirect range restriction
Summary of meta-analysis correcting each correlation individually
Exercise 1: Bare bones meta-analysis
Exercise 2: Meta-analysis correcting each correlation individually
Meta-Analysis of Correlations Using Artifact Distributions
Full artifact distribution meta-analysis
Accuracy of corrections for artifacts
Mixed meta-analysis: Partial artifact information in individual studies
Summary of artifact distribution of meta-analysis of correlations
Exercise: Artifact distribution meta-analysis
Technical Questions in Meta-Analysis of Correlations
r versus : Which should be used?
r vs. regression slopes and intercepts in meta-analysis
Technical factors that cause overestimation of
Fixed and random models in meta-analysis
Credibility vs. confidence intervals in meta-analysis
Computing confidence intervals in meta-analysis
Range Restriction in meta-analysis: New technical analysis
Criticisms of meta-analysis procedures for correlations
Meta-Analysis of Experimental Effects and Other Dichotomous Comparisons
Treatment Effects: Experimental Artifacts and Their Impact
Quantification of the treatment effect: The d statistic and the point-biserial correlation
Sampling error in d values: Illustrations
Error of measurement in the dependent variable
Error of measurement in the treatment variable
Variation across studies in treatment strength
Range variation on the dependent variable
Dichotomization of the dependent variable
Imperfect construct validity in the dependent variable
Imperfect construct validity in the treatment variable
Bias in the effect size (d statistic)
Recording, computational, and transcriptional errors
Multiple artifacts and corrections
Meta-Analysis Methods for d Values
Effect size indices: d and r
An Alternative to d: Glass' d
Sampling error in the d statistic
Cumulation and correction of the variance for sampling error
Analysis of moderator variables
Correcting d values for measurement error in the dependent variable
Measurement error in the independent variable in experiments
Other artifacts and their effects
Correcting for multiple artifacts
Summary of meta-analysis of d values
Exercise: Meta-Analysis of d-Values
Technical Questions in Meta-Analysis of d Values
Alternative experimental designs
Within-subjects experimental designs
Meta-analysis and the within-subjects design
Statistical power in the two designs
Threats to internal and external validity
Bias in observed d values
Use of multiple regression in moderation analysis of d values
General Issues in Meta-Analysis
General Technical Issues in Meta-analysis
Fixed effects versus random effects models in meta-analysis
Second order sampling error: General principles
Detecting moderators not hypothesized a priori
Second order meta-analysis
Large N studies and meta-analysis
Second order sampling error: Technical treatment
The detection of moderator variables: Summary
Hierarchical analysis of moderator variables
Exercise: Second order meta-analysis
Cumulation of Findings Within Studies
Fully replicated designs
Conceptual replications
Conceptual replications and confirmatory factor analysis
Conceptual replications: A alternative approach
Analysis of subgroups
Summary
Methods of Integrating Findings Across Studies and Related Software
The traditional narrative procedure
The traditional voting method
Cumulation of p-values across studies
Statistically correct vote counting procedures
Meta-analysis of research studies
Unresolved problems in meta-analysis
Summary of methods of integrating studies
Computer programs for meta-analysis
Locating, Evaluating, and Coding Studies
Conducting a thorough literature search
What to do about studies with "methodological weaknesses"
Coding studies in meta-analysis
What to include in the meta-analysis report
Information needed in reports of primary studies
Appendix: Coding sheet for validity studies
Availability and Source Bias in Meta-Analysis
Some evidence on bias
Effects of methodological quality on mean effect sizes from different sources
Multiple hypotheses and other considerations in availability bias
Methods for detecting availability bias
Methods for correcting for availability bias
Summary of Psychometric Meta-Analysis
Meta-analysis methods and theories of data
What is the ultimate purpose of meta-analysis?
Appendix: Windows Based Meta-Analysis Software Package
References
Author Index
Subject Index
About the Authors
Preface to 1st Edition
Acknowledgements
Introduction to Meta-Analysis
Integration Research Findings Across Studies
General problem and an example
Problems with statistical significance tests
Is statistical power the solution?
Confidence intervals
Meta-analysis
Role of meta-analysis in the behavioral and social sciences
Role of meta-analysis in theory development
Increasing use of meta-analysis
Meta-analysis in industrial-organizational psychology
Wider impact of meta-analysis on psychology
Impact of meta-analysis outside psychology
Meta-analysis and social policy
Meta-analysis and theories of data
Conclusions
Study Artifacts and Their Impact on Study Outcomes
Study Artifacts
Sampling error, statistical power, and the interpretation of research literatures
When and how to cumulate
Undercorrection for artifacts in the corrected standard deviation
Coding study characteristics and capitalization on sampling error in moderator analysis
A look ahead in the book
Meta-Analysis of Correlations
Meta-Analysis of Correlations Corrected Individually for Artifacts
Introduction and Overview
Bare bones meta-analysis: Correcting for sampling error only
Artifacts other than sampling error
Multiple simultaneous artifacts
Meta-analysis of individually corrected correlations
A worked example: Indirect range restriction
Summary of meta-analysis correcting each correlation individually
Exercise 1: Bare bones meta-analysis
Exercise 2: Meta-analysis correcting each correlation individually
Meta-Analysis of Correlations Using Artifact Distributions
Full artifact distribution meta-analysis
Accuracy of corrections for artifacts
Mixed meta-analysis: Partial artifact information in individual studies
Summary of artifact distribution of meta-analysis of correlations
Exercise: Artifact distribution meta-analysis
Technical Questions in Meta-Analysis of Correlations
r versus : Which should be used?
r vs. regression slopes and intercepts in meta-analysis
Technical factors that cause overestimation of
Fixed and random models in meta-analysis
Credibility vs. confidence intervals in meta-analysis
Computing confidence intervals in meta-analysis
Range Restriction in meta-analysis: New technical analysis
Criticisms of meta-analysis procedures for correlations
Meta-Analysis of Experimental Effects and Other Dichotomous Comparisons
Treatment Effects: Experimental Artifacts and Their Impact
Quantification of the treatment effect: The d statistic and the point-biserial correlation
Sampling error in d values: Illustrations
Error of measurement in the dependent variable
Error of measurement in the treatment variable
Variation across studies in treatment strength
Range variation on the dependent variable
Dichotomization of the dependent variable
Imperfect construct validity in the dependent variable
Imperfect construct validity in the treatment variable
Bias in the effect size (d statistic)
Recording, computational, and transcriptional errors
Multiple artifacts and corrections
Meta-Analysis Methods for d Values
Effect size indices: d and r
An Alternative to d: Glass' d
Sampling error in the d statistic
Cumulation and correction of the variance for sampling error
Analysis of moderator variables
Correcting d values for measurement error in the dependent variable
Measurement error in the independent variable in experiments
Other artifacts and their effects
Correcting for multiple artifacts
Summary of meta-analysis of d values
Exercise: Meta-Analysis of d-Values
Technical Questions in Meta-Analysis of d Values
Alternative experimental designs
Within-subjects experimental designs
Meta-analysis and the within-subjects design
Statistical power in the two designs
Threats to internal and external validity
Bias in observed d values
Use of multiple regression in moderation analysis of d values
General Issues in Meta-Analysis
General Technical Issues in Meta-analysis
Fixed effects versus random effects models in meta-analysis
Second order sampling error: General principles
Detecting moderators not hypothesized a priori
Second order meta-analysis
Large N studies and meta-analysis
Second order sampling error: Technical treatment
The detection of moderator variables: Summary
Hierarchical analysis of moderator variables
Exercise: Second order meta-analysis
Cumulation of Findings Within Studies
Fully replicated designs
Conceptual replications
Conceptual replications and confirmatory factor analysis
Conceptual replications: A alternative approach
Analysis of subgroups
Summary
Methods of Integrating Findings Across Studies and Related Software
The traditional narrative procedure
The traditional voting method
Cumulation of p-values across studies
Statistically correct vote counting procedures
Meta-analysis of research studies
Unresolved problems in meta-analysis
Summary of methods of integrating studies
Computer programs for meta-analysis
Locating, Evaluating, and Coding Studies
Conducting a thorough literature search
What to do about studies with "methodological weaknesses"
Coding studies in meta-analysis
What to include in the meta-analysis report
Information needed in reports of primary studies
Appendix: Coding sheet for validity studies
Availability and Source Bias in Meta-Analysis
Some evidence on bias
Effects of methodological quality on mean effect sizes from different sources
Multiple hypotheses and other considerations in availability bias
Methods for detecting availability bias
Methods for correcting for availability bias
Summary of Psychometric Meta-Analysis
Meta-analysis methods and theories of data
What is the ultimate purpose of meta-analysis?
Appendix: Windows Based Meta-Analysis Software Package
References
Author Index
Subject Index
About the Authors