How to Read a Paper
Description
Learn to assess published research in this best-selling introduction to evidence-based healthcare
Evidence-based practices have revolutionized medical care. Clinical and scientific papers have something to offer practitioners at every level of the profession, from students to established clinicians in medicine, nursing and allied professions. Novices are often intimidated by the idea of reading and appraising the research literature. How to Read a Paper demystifies this process with a thorough, engaging introduction to how clinical research papers are constructed and how to evaluate them. Now fully updated to incorporate new areas of research, readers of the seventh edition of How to Read a Paper will also find:
- A careful balance between the principles of evidence-based healthcare and clinical practice
- New chapters covering consensus methods, mechanistic evidence, big data and artificial intelligence
- Detailed coverage of subjects like assessing methodological quality, systemic reviews and meta-analyses, qualitative research, and more.
How to Read a Paper is ideal for all healthcare students and professionals seeking an accessible introduction to evidence-based healthcare - particularly those sitting undergraduate and postgraduate exams and preparing for interviews.
More details
Other editions
Additional editions
Persons
Trisha Greenhalgh is a general practitioner and Professor of Primary Care Health Sciences and Fellow of Green Templeton College at the University of Oxford.
Paul Dijkstra is a sport and exercise medicine physician and Director of Medical Education at Aspetar Orthopaedic and Sports Medicine Hospital in Doha, Qatar. He has an academic affiliation with the Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences at the University of Oxford.
Content
Foreword to the first edition by Professor Sir David Weatherall xii
Preface to the seventh edition xiv
Preface to the first edition xvii
Acknowledgements xix
Chapter 1 Why read papers at all? 1
Does 'evidence- based medicine' simply mean 'reading papers in medical journals'? 1
Why do people sometimes groan when you mention evidence- based healthcare? 4
Before you start: formulate the problem 11
Exercises based on this chapter 13
References 14
Chapter 2 Searching the literature 15
The information jungle 15
What are you looking for? 16
Levels upon levels of evidence 17
Synthesised sources: systems, summaries and syntheses 18
Pre-appraised sources: synopses of systematic reviews and primary studies 21
Specialised resources 22
Primary studies: tackling the jungle 23
One-stop shopping: federated search engines 25
Using artificial intelligence to search the literature 25
Asking for help and asking around 26
Online tutorials for effective searching 26
Exercises based on this chapter 27
References 28
Chapter 3 Getting your bearings: what is this paper about? 30
The science of 'trashing' papers 30
Three preliminary questions to get your bearings 32
What are randomised controlled trials and why do they matter? 34
What are cohort studies? 38
What are case-control studies? 40
What are cross-sectional surveys? 40
What are case reports? 41
The traditional hierarchy of evidence 42
Exercises based on this chapter 43
References 43
Chapter 4 Assessing methodological quality 45
Was the study original? 45
Who is the study about? 46
Was the design of the study sensible? 47
Was bias avoided or minimised? 49
Was assessment 'blind'? 54
Were preliminary statistical questions addressed? 55
A note on ethical considerations 58
Summing up 59
Exercises based on this chapter 60
References 60
Chapter 5 Statistics for the non-statistician 63
How can non-statisticians evaluate statistical tests? 63
Have the authors set the scene correctly? 65
Paired data, tails and outliers 71
Correlation, regression and causation 72
Probability and confidence 74
The bottom line (quantifying the chance of benefit and harm) 77
Summary 79
Exercises based on this chapter 79
References 80
Chapter 6 Papers that report clinical trials of simple interventions 82
What is a clinical trial? 82
Drug trials: 'evidence' and marketing 83
Making decisions about therapy 86
Surrogate endpoints 87
What information to expect in a paper describing a randomised controlled trial: the CONSORT statement 91
Getting worthwhile evidence from pharmaceutical representatives 91
A note on vaccine trials 94
Exercises based on this chapter 95
References 95
Chapter 7 Papers that report trials of complex interventions 99
Complex interventions 99
Ten questions to ask about a paper describing a complex intervention 101
Exercises based on this chapter 106
References 107
Chapter 8 Papers that report diagnostic or screening tests 109
Ten suspects in the dock 109
Validating diagnostic tests against a gold standard 110
Ten questions to ask about a paper that claims to validate a diagnostic or screening test 115
Likelihood ratios 119
Clinical prediction models 122
Exercises based on this chapter 124
References 125
Chapter 9 Papers that summarise other papers (systematic reviews and meta-analyses) 128
When is a review systematic? 128
Evaluating systematic reviews: five questions to ask 131
Meta-analysis for the non-statistician 137
Explaining heterogeneity 142
New approaches to systematic review 145
Exercises based on this chapter 146
References 146
Chapter 10 Papers that advise you what to do (guidelines) 151
The great guidelines debate 151
Ten questions to ask about a clinical guideline 155
Exercises based on this chapter 162
References 162
Chapter 11 Papers that estimate what things cost (health economic evaluations) 164
What is an economic evaluation? 164
Health economics studies: two key approaches 166
Costs and benefits of health interventions 167
Measuring the value of health states 168
Quality-adjusted life-years 169
Low-value health: choosing wisely 171
Twelve questions to ask about a health economic evaluation 172
Conclusion 176
Exercises based on this chapter 176
References 177
Chapter 12 Papers that go beyond numbers (qualitative research) 179
What is qualitative research? 179
Summarising and synthesising qualitative research 183
Nine questions to ask about a qualitative research paper 184
Conclusion 191
Exercises based on this chapter 192
References 192
Chapter 13 Papers that report questionnaire research 195
The rise and rise of questionnaire research 195
Ten questions to ask about a paper describing a questionnaire study 196
Exercises based on this chapter 205
References 206
Chapter 14 Papers that report quality improvement case studies 208
What are quality improvement studies and how should we research them? 208
Ten questions to ask about a paper describing a quality improvement initiative 210
Conclusion 217
Exercises based on this chapter 217
References 218
Chapter 15 Papers that describe genetic association studies 220
The three eras of human genetic studies (so far) 220
What is a genome-wide association study? 222
Clinical applications of genome-wide association studies 225
Direct- to- consumer genetic testing 226
Mendelian randomisation studies 227
Epigenetics: a space to watch 228
Ten questions to ask about a genetic association study 230
Exercises based on this chapter 234
References 234
Chapter 16 Applying evidence with patients 237
The patient perspective 237
Patient- reported outcome measures 239
Shared decision- making 240
Option grids 243
n-of-1 trials and other individualised approaches 244
Exercises based on this chapter 246
References 247
Contents xi
Chapter 17 Papers on artificial intelligence in healthcare 249
Introduction 249
Artificial intelligence 251
Big data 253
Machine learning 254
Generative artificial intelligence: large language and multimodal models 254
Ethical principles for the use of artificial intelligence for health 255
Appraising artificial intelligence papers: a plethora of checklists 256
Ten questions to ask about a paper that reports AI studies in healthcare 260
Summary 264
Exercises based on this chapter 264
References 265
Chapter 18 EBM+: the importance of mechanistic evidence 268
What is mechanistic evidence? An example 268
The many types of mechanistic evidence and a preliminary hierarchy 269
EBM+ means 'both and', not 'either or' 270
Mechanistic evidence in the COVID-19 pandemic 272
Exercises based on this chapter 275
References 276
Chapter 19 Papers that report consensus exercises 278
Why are consensus method papers important? 279
How do experts choose and reach consensus on a specific topic? 279
Consensus methods 281
Ten questions to ask about a paper that reports a consensus statement 285
Exercises based on this chapter 290
References 291
Chapter 20 Criticisms of evidence-based healthcare 293
What's wrong with evidence-based healthcare when it's done badly? 293
What's wrong with evidence-based healthcare when it's done well? 296
Why is 'evidence-based policymaking' so hard to achieve? 299
Exercises based on this chapter 301
References 301
Appendix 1 Checklists for finding, appraising and implementing evidence 304
Appendix 2 Assessing the effects of an intervention 316
Index 317
Chapter 1
Why read papers at all?
Does 'evidence-based medicine' simply mean 'reading papers in medical journals'?
Evidence-based medicine (EBM), which is part of the broader field of evidence-based healthcare (EBHC), is much more than just reading papers. According to what is still (more than 25 years after it was written) the most widely quoted definition, it is 'the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients' [1]. This definition is useful up to a point, but it misses out a very important aspect of the subject - and that is the use of mathematics. Even if you know almost nothing about EBHC, you probably know it talks a lot about numbers and ratios! A few years ago, Trisha and Anna Donald decided to be upfront about this in our own teaching, and proposed this alternative definition:
Evidence-based medicine is the use of mathematical estimates of the risk of benefit and harm, derived from high-quality research on population samples, to inform clinical decision-making in the diagnosis, investigation or management of individual patients.
The defining feature of EBHC, then, is the use of numbers derived from research on population samples to inform decisions about individuals. This, of course, begs the question 'What is research?' - for which a reasonably accurate answer might be 'Focused, systematic enquiry aimed at generating new knowledge'. In later chapters, we explain how this definition can help you distinguish genuine research (which should inform your practice) from the poor-quality endeavours of well-meaning amateurs (which you should politely ignore). (As an aside, it has become fashionable to include qualitative research within EBHC, and we do cover this in chapter 12, but most people talking about EBM and EBHC are referring to research that generates numbers).
If you follow an evidence-based approach to clinical decision-making, therefore, all sorts of issues relating to your patients (or, if you work in public health medicine, issues relating to groups of people) will prompt you to ask questions about scientific evidence, seek answers to those questions in a systematic way and alter your practice accordingly.
You might ask questions, for example, about a patient's symptoms ('In a 34-year-old man with left-sided chest pain, what is the probability that there is a serious heart problem, and, if there is, will it show up on a resting ECG?'), about physical or diagnostic signs ('In an otherwise uncomplicated labour, does the presence of meconium [indicating fetal bowel movement] in the amniotic fluid indicate significant deterioration in the physiological state of the fetus?'), about the prognosis of an illness ('If a previously well two-year-old has a short fit associated with a high temperature, what is the chance that she will subsequently develop epilepsy?'), about therapy ('In patients with acute coronary syndrome [heart attack], are the risks associated with thrombolytic drugs [clot busters] outweighed by the benefits, whatever the patient's age, sex and ethnic origin?'), about cost-effectiveness ('Is the cost of this new anti-cancer drug justified, compared with other ways of spending limited healthcare resources?'), about patients' preferences ('In an 87-year-old woman with intermittent atrial fibrillation and a recent transient ischaemic attack, do the potential harms and inconvenience of thrombolytic therapy outweigh the risks of not taking it?') and about a host of other aspects of health and health services.
Professor Sackett, in the opening editorial of the very first issue of the journal Evidence-Based Medicine, summarised the essential steps in the emerging science of EBM [2]:
- Convert our information needs into answerable questions (i.e. to formulate the problem).
- Track down the best evidence with which to answer these questions - which may come from the clinical examination, the diagnostic laboratory, the published literature or other sources.
- Appraise the evidence critically (i.e. weigh it up) to assess its validity (closeness to the truth) and usefulness (clinical applicability).
- Implement the results of this appraisal in our clinical practice.
- Evaluate our performance.
Hence, EBHC requires you not only to read papers but to read the right papers at the right time, and then to alter your behaviour (and, what is often more difficult, influence the behaviour of other people) in the light of what you have found. Sometimes, how-to-do-it courses in EBHC concentrate too heavily on the third of these five steps (critical appraisal) to the exclusion of all the others. Yet, if you have asked the wrong question or sought answers from the wrong sources, you might as well not read any papers at all. And all your training in search techniques and critical appraisal will go to waste if you do not put at least as much effort into implementing valid evidence and measuring progress towards your goals as you do into reading the paper. A few years ago, Trisha added three more stages to Sackett's five-stage model to incorporate the patient's perspective: the resulting eight stages, producing a context-sensitive checklist for evidence-based practice, which (like the other checklists in this book) is given in Appendix 1.
If we were to be pedantic about the title of this book, these broader aspects of EBHC should not even get a mention here. But we hope you understand that the book would be incomplete without the final section of this chapter (Before you start: formulate the problem), Chapter 2 (Searching the literature), and Chapter 16 (Applying evidence with patients). Chapters 3-15 describe step three of the EBHC process: critical appraisal; that is, what you should do when you actually have the paper in front of you. Chapter 20 deals with common criticisms of EBHC. The challenges of implementation are so complex that they needed a book of their own, How to Implement Evidence-Based Healthcare [3].
If you want to explore the subject of EBHC on the Internet, you could try the websites listed in Box 1.1 (these were the top suggestions when we asked our X [formerly Twitter] followers which ones they found most useful). If you're not ready for that yet, don't worry at this stage, but do put learning to use web-based resources on your to-do list. Don't worry either when you discover that there are over 1000 websites dedicated to EBM and EBHC; they all offer very similar material and you certainly don't need to visit them all.
Box 1.1 Web-based resources for evidence-based medicine
BMJ Evidence-Based Medicine Toolkit: a resource site maintained by this leading UK medical journal containing a wealth of resources and links for EBM, including links to critical appraisal checklists and statistical tools. https://bestpractice.bmj.com/info/toolkit
National Institute for Health and Care Excellence: this UK-based website, which is also popular outside the UK, links to evidence-based guidelines and topic reviews. www.nice.org.uk
The A-Z List of Evidence-Based Medicine Resources: A one-stop shop for various databases maintained by Dartmouth Libraries at Dartmouth College, Hanover, NH, USA, including PubMed, the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effectiveness (DARE): https://www.dartmouth.edu/library/biomed/guides/research/ebm-az-list.html
Why do people sometimes groan when you mention evidence-based healthcare?
Critics of EBHC might define it as 'the tendency of a group of young, confident and highly numerate medical academics to belittle the performance of experienced clinicians using a combination of epidemiological jargon and statistical sleight of hand' or 'the argument, usually presented with near-evangelistic zeal, that no health-related action should ever be taken by a doctor, a nurse, a purchaser of health services or a policymaker unless and until the results of several large and expensive research trials have appeared in print and approved by a committee of experts'.
Anyone who works face to face with patients knows how often it is necessary to seek new information before making a clinical decision. In general, we don't put a patient on a drug without evidence that it is likely to work. Apart from anything else, such off-licence use of medication is, strictly speaking, illegal. Surely we have all been practising EBHC for years?
Well, no, we haven't. There have been a number of surveys on the behaviour of doctors, nurses and related professionals and, while things seem to be improving, performance still falls short. It was estimated in the 1970s in the United States that only around 10-20% of all health technologies then available (i.e. drugs, procedures, operations, etc.) were evidence-based; that estimate improved to 21% in 1990. Studies of the interventions offered to consecutive series of...
