Individual Participant Data Meta-Analysis
Description
Split into five parts, the book chapters take the reader through the journey from initiating and planning IPD projects to obtaining, checking, and meta-analysing IPD, and appraising and reporting findings. The book initially focuses on the synthesis of IPD from randomised trials to evaluate treatment effects, including the evaluation of participant-level effect modifiers (treatment-covariate interactions). Detailed extension is then made to specialist topics such as diagnostic test accuracy, prognostic factors, risk prediction models, and advanced statistical topics such as multivariate and network meta-analysis, power calculations, and missing data.
Intended for a broad audience, the book will enable the reader to:
* Understand the advantages of the IPD approach and decide when it is needed over a conventional systematic review
* Recognise the scope, resources and challenges of IPD meta-analysis projects
* Appreciate the importance of a multi-disciplinary project team and close collaboration with the original study investigators
* Understand how to obtain, check, manage and harmonise IPD from multiple studies
* Examine risk of bias (quality) of IPD and minimise potential biases throughout the project
* Understand fundamental statistical methods for IPD meta-analysis, including two-stage and one-stage approaches (and their differences), and statistical software to implement them
* Clearly report and disseminate IPD meta-analyses to inform policy, practice and future research
* Critically appraise existing IPD meta-analysis projects
* Address specialist topics such as effect modification, multiple correlated outcomes, multiple treatment comparisons, non-linear relationships, test accuracy at multiple thresholds, multiple imputation, and developing and validating clinical prediction models
Detailed examples and case studies are provided throughout.
More details
Other editions
Additional editions
Persons
Richard D. Riley is Professor of Biostatistics in the School of Medicine, Keele University, UK.
Jayne F. Tierney is Professor of Evidence Synthesis at the MRC Clinical Trials Unit, University College London, UK.
Lesley A. Stewart is Professor of Evidence Synthesis and Director of the Centre for Reviews and Dissemination, University of York, UK.
Content
Acknowledgements xxiii
1 Individual Participant Data Meta-analysis for Healthcare Research 1
Richard D. Riley, Lesley A. Stewart, and Jayne F. Tierney
1.1 Introduction 1
1.2 What Is IPD and How Does It Differ from Aggregate Data? 1
1.3 IPD Meta-analysis: A New Era for Evidence Synthesis 2
1.4 Scope of This Book and Intended Audience 2
Part I Rationale, Planning, and Conduct 7
2 Rationale for Embarking on an IPD Meta-analysis Project 9
Jayne F. Tierney, Richard D. Riley, Catrin Tudur Smith, Mike Clarke, and Lesley A. Stewart
2.1 Introduction 9
2.2 How Does the Research Process Differ for IPD and Aggregate Data Meta-analysis Projects? 10
2.3 What Are the Potential Advantages of an IPD Meta-analysis Project? 11
2.4 What Are the Potential Challenges of an IPD Meta-Analysis Project? 14
2.5 Empirical Evidence of Differences between Results of IPD and Aggregate Data Metaanalysis Projects 14
2.6 Guidance for Deciding When IPD Meta-analysis Projects Are Needed to Evaluate Treatment Effects from Randomised Trials 15
2.7 Concluding Remarks 19
3 Planning and Initiating an IPD Meta-analysis Project 21
Lesley A. Stewart, Richard D. Riley, and Jayne F. Tierney
3.1 Introduction 22
3.2 Organisational Approach 22
3.3 Developing a Project Scope 26
3.4 Assessing Feasibility and 'In Principle' Support and Collaboration 26
3.5 Establishing a Team with the Right Skills 29
3.6 Advisory and Governance Functions 30
3.7 Estimating How Long the Project Will Take 31
3.8 Estimating the Resources Required 33
3.9 Obtaining Funding 38
3.10 Obtaining Ethical Approval 39
3.11 Data-sharing Agreement 41
3.12 Additional Planning for Prospective Meta-analysis Projects 41
3.13 Concluding Remarks 43
4 Running an IPD Meta-analysis Project: From Developing the Protocol to Preparing Data for Meta-analysis 45
Jayne F. Tierney, Richard D. Riley, Larysa H.M. Rydzewska, and Lesley A. Stewart
4.1 Introduction 46
4.2 Preparing to Collect IPD 46
4.3 Initiating and Maintaining Collaboration 57
4.4 Obtaining IPD 59
4.5 Checking and Harmonising Incoming IPD 62
4.6 Checking the IPD to Inform Risk of Bias Assessments 66
4.7 Assessing and Presenting the Overall Quality of a Trial 76
4.8 Verification of Finalised Trial IPD 77
4.9 Merging IPD Ready for Meta-analysis 77
4.10 Concluding Remarks 80
Part I References 81
Part II Fundamental Statistical Methods and Principles 87
5 The Two-stage Approach to IPD Meta-analysis 89
Richard D. Riley, Thomas P.A. Debray, Tim P. Morris, and Dan Jackson
5.1 Introduction 90
5.2 First Stage of a Two-stage IPD Meta-analysis 90
5.3 Second Stage of a Two-stage IPD Meta-analysis 106
5.4 Meta-regression and Subgroup Analyses 120
5.5 The ipdmetan Software Package 121
5.6 Combining IPD with Aggregate Data from non-IPD Trials 124
5.7 Concluding Remarks 125
6 The One-stage Approach to IPD Meta-analysis 127
Richard D. Riley and Thomas P.A. Debray 127
6.1 Introduction 128
6.2 One-stage IPD Meta-analysis Models Using Generalised Linear Mixed Models 129
6.3 One-stage Models for Time-to-event Outcomes 152
6.4 One-stage Models Combining Different Sources of Evidence 159
6.5 Reporting of One-stage Models in Protocols and Publications 162
6.6 Concluding Remarks 162
7 Using IPD Meta-analysis to Examine Interactions between Treatment Effect and Participant-level Covariates 163
Richard D. Riley and David J. Fisher
7.1 Introduction 164
7.2 Meta-regression and Its Limitations 166
7.3 Two-stage IPD Meta-analysis to Estimate Treatment-covariate Interactions 168
7.4 The One-stage Approach 174
7.5 Combining IPD and non-IPD Trials 181
7.6 Handling of Continuous Covariates 184
7.7 Handling of Categorical or Ordinal Covariates 191
7.8 Misconceptions and Cautions 191
7.9 Is My Identified Treatment-covariate Interaction Genuine? 195
7.10 Reporting of Analyses of Treatment-covariate Interactions 196
7.11 Can We Predict a New Patient's Treatment Effect? 196
7.11.1 Linking Predictions to Clinical Decision Making 198
7.12 Concluding Remarks 198
8 One-stage versus Two-stage Approach to IPD Meta-analysis: Differences and Recommendations 199
Richard D. Riley, Danielle L. Burke, and Tim Morris
8.1 Introduction 200
8.2 One-stage and Two-stage Approaches Usually Give Similar Results 200
8.3 Ten Key Reasons Why One-stage and Two-stage Approaches May Give Different Results 203
8.4 Recommendations and Guidance 216
8.5 Concluding Remarks 217
Part II References 219
Part III Critical Appraisal and Dissemination 237
9 Examining the Potential for Bias in IPD Meta-analysis Results 239
Richard D. Riley, Jayne F. Tierney, and Lesley A. Stewart
9.1 Introduction 240
9.2 Publication and Reporting Biases of Trials 240
9.3 Biased Availability of the IPD from Trials 244
9.4 Trial Quality (risk of bias) 247
9.5 Other Potential Biases Affecting IPD Meta-analysis Results 248
9.6 Concluding Remarks 251
10 Reporting and Dissemination of IPD Meta-analyses 253
Lesley A. Stewart, Richard D. Riley, and Jayne F. Tierney
10.1 Introduction 253
10.2 Reporting IPD Meta-analysis Projects in Academic Reports 254
10.3 Additional Means of Disseminating Findings 266
10.4 Concluding Remarks 270
11 A Tool for the Critical Appraisal of IPD Meta-analysis Projects (CheckMAP) 271
Jayne F. Tierney, Lesley A. Stewart, Claire L. Vale, and Richard D. Riley
11.1 Introduction 271
11.2 The CheckMAP Tool 272
11.3 Was the IPD Meta-analysis Project Done within a Systematic Review Framework? 272
11.4 Were the IPD Meta-analysis Project Methods Pre-specified in a Publicly Available Protocol? 274
11.5 Did the IPD Meta-analysis Project Have a Clear Research Question Qualified by Explicit Eligibility Criteria? 276
11.6 Did the IPD Meta-analysis Project Have a Systematic and Comprehensive Search Strategy? 276
11.7 Was the Approach to Data Collection Consistent and Thorough? 277
11.8 Were IPD Obtained from Most Eligible Trials and Their Participants? 277
11.9 Was the Validity of the IPD Checked for Each Trial? 278
11.10 Was the Risk of Bias Assessed for Each Trial and Its Associated IPD? 27811.10.1 Was the Randomisation Process Checked Based on IPD? 278
11.11 Were the Methods of Meta-analysis Appropriate? 280
11.12 Concluding Remarks 283
Part III References 285
Part IV Special Topics in Statistics 291
12 Power Calculations for Planning an IPD Meta-analysis 293
Richard D. Riley and Joie Ensor
12.1 Introduction 294
12.2 Motivating Example: Power of a Planned IPD Meta-analysis of Trials of Interventions to Reduce Weight Gain in Pregnant Women 295
12.3 The Contribution of Individual Trials Toward Power 301
12.4 The Impact of Model Assumptions on Power 302
12.5 Extensions 305
12.6 Concluding Remarks 309
13 Multivariate Meta-analysis Using IPD 311
Richard D. Riley, Dan Jackson, and Ian R. White
13.1 Introduction 312
13.2 General Two-stage Approach for Multivariate IPD Meta-analysis 314
13.3 Application to an IPD Meta-analysis of Anti-hypertensive Trials 329
13.4 Extension to Multivariate Meta-regression 333
13.5 Potential Limitations of Multivariate Meta-analysis 334
13.6 One-stage Multivariate IPD Meta-analysis Applications 337
13.7 Special Applications of Multivariate Meta-analysis 340
13.8 Concluding Remarks 346
14 Network Meta-analysis Using IPD 347
Richard D. Riley, David M. Phillippo, and Sofia Dias
14.1 Introduction 348
14.2 Rationale and Assumptions for Network Meta-analysis 348
14.3 Network Meta-analysis Models Assuming Consistency 350
14.4 Ranking Treatments 357
14.5 How Do We Examine Inconsistency between Direct and Indirect Evidence? 359
14.6 Benefits of IPD for Network Meta-analysis 361
14.7 Combining IPD and Aggregate Data in Network Meta-analysis 365
14.8 Further Topics 370
14.9 Concluding Remarks 372
Part IV References 375
Part V Diagnosis, Prognosis and Prediction 387
15 IPD Meta-analysis for Test Accuracy Research 389
Richard D. Riley, Brooke Levis, and Yemisi Takwoingi 389
15.1 Introduction 390
15.2 Motivating Example: Diagnosis of Fever in Children Using Ear Temperature 394
15.3 Key Steps Involved in an IPD Meta-analysis of Test Accuracy Studies 397
15.4 IPD Meta-analysis of Test Accuracy at Multiple Thresholds 410
15.5 IPD Meta-analysis for Examining a Test's Clinical Utility 414
15.6 Comparing Tests 418
15.7 Concluding Remarks 420
16 IPD Meta-analysis for Prognostic Factor Research 421
Richard D. Riley, Karel G.M. Moons, and Thomas P.A. Debray
16.1 Introduction 422
16.2 Potential Advantages of an IPD Meta-analysis 424
16.3 Key Steps Involved in an IPD Meta-analysis of Prognostic Factor Studies 427
16.4 Software 444
16.5 Concluding Remarks 444
17 IPD Meta-analysis for Clinical Prediction Model Research 447
Richard D. Riley, Kym I.E. Snell, Laure Wynants, Valentijn M.T. de Jong, Karel G.M. Moons, and Thomas P.A. Debray
17.1 Introduction 448
17.2 IPD Meta-analysis for Prediction Model Research 448
17.3 External Validation of an Existing Prediction Model Using IPD Meta-analysis 455
17.4 Updating and Tailoring of a Prediction Model Using IPD Meta-analysis 470
17.5 Comparison of Multiple Existing Prediction Models Using IPD Meta-analysis 472
17.6 Using IPD Meta-analysis to Examine the Added Value of a New Predictor to an Existing Prediction Model 478
17.7 Developing a New Prediction Model Using IPD Meta-analysis 479
17.8 Examining the Utility of a Prediction Model Using IPD Meta-analysis 491
17.9 Software 494
17.10 Reporting 495
17.11 Concluding Remarks 495
18 Dealing with Missing Data in an IPD Meta-analysis 499
Thomas Debray, Kym I.E. Snell, Matteo Quartagno, Shahab Jolani, Karel G.M. Moons, and Richard D. Riley 499
18.1 Introduction 500
18.2 Motivating Example: IPD Meta-analysis Validating Prediction Models for Risk of Preeclampsia in Pregnancy 500
18.3 Types of Missing Data in an IPD Meta-analysis 502
18.4 Recovering Actual Values of Missing Data within IPD 502
18.5 Mechanisms and Patterns of Missing Data in an IPD Meta-analysis 502
18.6 Multiple Imputation to Deal with Missing Data in a Single Study 506
18.7 Ensuring Congeniality of Imputation and Analysis Models 509
18.8 Dealing with Sporadically Missing Data in an IPD Meta-analysis by Applying Multiple Imputation for Each Study Separately 509
18.9 Dealing with Systematically Missing Data in an IPD Meta-analysis Using a Bivariate Metaanalysis of Partially and Fully Adjusted Results 511
18.10 Dealing with Both Sporadically and Systematically Missing Data in an IPD Meta-analysis Using Multilevel Modelling 514
18.11 Comparison of Methods and Recommendations 521
18.12 Software 523
18.13 Concluding Remarks 524
Part V References 525
Index 000
1
Individual Participant Data Meta-Analysis for Healthcare Research
Richard D. Riley, Lesley A. Stewart, and Jayne F. Tierney
1.1 Introduction
Healthcare and clinical decision-making should be guided by the evidence arising from high-quality research studies. Often a single study is insufficient to make firm recommendations, and so multiple studies are conducted to address the same research question. This motivates the need for evidence synthesis: the combination of data from multiple studies to provide an overall summary of current knowledge. For example, when multiple randomised trials have examined the effect of a particular treatment, evidence syntheses are needed to combine and summarise the information from these trials, in order to establish whether the treatment is effective or not.
Systematic reviews are the cornerstone of evidence synthesis and evidence-based decision-making in healthcare. They use transparent methods to identify, appraise and combine a body of research evidence, with the goal of producing summary results that guide best practices for stakeholders including patients, clinicians, health professionals, and policy-makers. Systematic review methodology has been championed by organisations such as Cochrane, who publish systematic reviews in the Cochrane Library summarising the effects of interventions,1 the accuracy of diagnostic tests,2 the prognostic effect of particular factors,3 and the performance of risk prediction models.4 Most systematic reviews include a meta-analysis,5 which is a statistical technique for combining (synthesising) quantitative data obtained from multiple research studies. Traditionally, most meta-analyses have used aggregate data extracted from study publications, but there is growing demand for meta-analyses that utilise individual participant data (IPD).6-9
This book is intended as a comprehensive handbook for healthcare researchers undertaking IPD meta-analysis projects. In this introductory chapter, we clarify differences between IPD and aggregate data, and outline why IPD meta-analysis projects are increasingly needed. Then, we detail the scope of our book and its intended audience, and signpost where to find material in subsequent chapters.
1.2 What Is IPD and How Does It Differ from Aggregate Data?
IPD refers to the raw information recorded for each participant in a research study (e.g. a randomised trial), such as baseline characteristics, prognostic factors, treatments received, outcomes and follow-up details, and can be represented by a dataset containing a separate row per participant and columns containing values for each participant-level variable. For example, IPD for a randomised trial of anti-hypertensive treatment will usually include the pre- and post-treatment blood pressure level, a treatment group indicator, important clinical characteristics and prognostic factors recorded at baseline (such as age, sex, BMI and comorbidities), and relevant follow-up information (such as time to cardiovascular disease or death). An IPD meta-analysis project, therefore, involves the collection, checking, harmonisation and synthesis of IPD from multiple studies to answer particular research questions. An excerpt of IPD collected from 10 randomised trials for an IPD meta-analysis project is given in Box 1.1(a), after harmonisation into a single dataset ready for meta-analysis to summarise the effect of anti-hypertensive treatment. This dataset contains a single row per participant in every trial.
In contrast, aggregate data refers to information averaged or estimated across all participants in a particular study, such as the treatment effect estimate, the total participants, and the mean age and proportion of males in each treatment group. Such aggregate data are derived from the IPD, and therefore the IPD can be considered the original source material. A conventional meta-analysis uses aggregate data (e.g. as extracted from study publications), rather than IPD. An example of aggregate data obtained from 10 randomised trials of anti-hypertensive treatment is shown in Box 1.1(b), after collation into a single dataset ready for meta-analysis. This dataset contains a single row per trial.
1.3 IPD Meta-Analysis: A New Era for Evidence Synthesis
"Data sharing is an important part of ensuring trust in research, and it should be the norm." 10
IPD meta-analysis projects began to emerge in the late 1980s and early 1990s,11,12 originating mainly in the cancer and cardiovascular disease fields.13 Calls to support IPD meta-analysis grew strongly throughout the 1990s alongside the formation of methodology working groups,8,14 in particular the Cochrane IPD Meta-Analysis Methods Group (https://methods.cochrane.org/ipdma/).7 In the decades since, the number of IPD meta-analysis projects has risen sharply (Figure 1.1). Early meta-analyses based on IPD were commonly described as overviews or pooled analyses,7,11,15,16 until IPD meta-analysis emerged as the preferred, and now most widely used, label. The IPD abbreviation initially referred to individual patient data, but now individual participant data is the more inclusive and accepted term.
The growth of IPD meta-analysis projects reflects their potential to revolutionise healthcare research,14,17 especially as they align with three major contemporary initiatives: reducing research waste,18 data sharing,19-24 and personalised healthcare.25,26 The sharing of IPD maximises the contribution of existing data from millions of research participants, and so is becoming an increasingly frequent stipulation of research funding. Leading medical journals now require data-sharing statements, with some even enforcing the sharing of IPD on request.23 This has led to dedicated data-sharing platforms and repositories being established to house IPD from existing studies.27-31 Furthermore, as the drive for personalised healthcare (also known as stratified or precision medicine) continues,25,26 researchers have recognised that, compared to using published aggregate data, IPD allows a more reliable evaluation of how participant-level characteristics are associated with outcome risk and response to treatment.32,33 Thus, IPD meta-analysis projects are now central to modern evidence synthesis in healthcare.
1.4 Scope of This Book and Intended Audience
Meta-Analysis Using Individual Participant Data: A Handbook for Healthcare Research provides a comprehensive introduction to the fundamental principles and methods that healthcare researchers need when considering, conducting or using IPD meta-analysis projects. Written and edited by researchers with substantial experience in the field, the book details key concepts and practical guidance alongside illustrated examples and summary learning points.
Box 1.1 Example of individual participant data (IPD) and how it differs from aggregate data
Illustrative example of 10 randomised trials examining the effect of anti-hypertensive treatment
(a) IPD
- The following table shows hypothetical IPD collected, checked and harmonised from 10 randomised trials examining the effect of anti-hypertensive treatment versus control in participants with hypertension.
- Each row provides the information for each participant in each trial, and each column provides participant-level information such as baseline characteristics and outcome values.
- Only a subset of the IPD is shown for brevity, as in reality many more rows and columns will be needed for each trial, to include all available participants and variables. Trial ID Participant ID Treatment group,
1 = treatment
0 = control Age
(years) SBP before treatment
(mmHg) SBP at 1 year
(mmHg) 1 1 1 46 137 111 1 2 1 35 143 133 (other rows for trial 1 omitted for brevity) 1 1454 0 62 209 219 2 1 0 55 170 155 2 2 1 38 144 139 (other rows for trial 2 omitted for brevity) 2 337 1 44 153 129 (rows for trials 3 to 9 omitted for brevity) 10 1 0 71 149 128 10 2 1 59 168 169 (other rows for trial 10 omitted for brevity) 10 4695 0 63 174 128 - This IPD can be used to...
