Handbook of Web Surveys
Description
The updated, must-have guide for creating and implementing web surveys
Revised and thoroughly updated, the second edition of Handbook of Web Surveys offers a practical and comprehensive guide for creating and conducting effective web surveys. The authors noted experts on the topic, include a review the Blaise system (which has been around for 30 years) and provide information on the most recent developments and techniques in the field. The book illustrates the steps needed to develop effective web surveys and explains how the survey process should be carried out. It also examines the aspects of sampling and presents a number of sampling designs.
The book includes ideas for overcoming possible errors in measurement and nonresponse. The authors also compare the various methods of data collection (face-to-face, telephone, mail, and mobile surveys) and discuss their advantages and disadvantages. Critical information for designing questionnaires for mobile devices is also provided. Filled with real-world examples, Handbook of Web Surveys discuss the key concepts, methods, and techniques of effective web surveys. This important book:
* Contains the most recent sampling designs and estimation procedures
* Offers ideas for overcoming errors in web surveys
* Includes information on mixed mode surveys
* Explores the concept of response probabilities
* Reviews all aspects of web panels
Written for researchers in government, business, economics, and social scientists, the second edition of Handbook of Web Surveys provides an introduction to web surveys and the various methods and techniques.
More details
Other editions
Additional editions
Persons
SILVIA BIFFIGNANDI is a professor at the Center for Statistics and Analysis of Sample Surveys, University of Bergamo, Bergamo, Italy.
JELKE BETHLEHEM is affiliated with Statistics Netherlands, a Division of Methodology and Quality, The Netherlands.
Content
Preface xi
1 The Road To Web Surveys 1
1.1 Introduction 1
1.2 Theory 2
1.2.1 The Everlasting Demand for Statistical Information 2
1.2.2 Traditional Data Collection 8
1.2.3 The Era of Computer-Assisted Interviewing 11
1.2.4 The Conquest of the Web 13
1.2.5 Web Surveys and Other Sources 23
1.2.6 Historic Summary 28
1.2.7 Present-Day Challenges and Opportunities 28
1.2.8 Conclusions from Modern-Day Challenges 30
1.2.9 Thriving in the Modern-Day Survey World 30
1.3 Application 31
1.3.1 Blaise 31
1.4 Summary 39
Key Terms 41
Exercises 42
References 44
2 About Web Surveys 47
2.1 Introduction 47
2.2 Theory 50
2.2.1 Typical Survey Situations 51
2.2.2 Why Online Data Collection? 56
2.2.3 Areas of Application 60
2.2.4 Trends in Web Surveys 62
2.3 Application 64
2.4 Summary 68
Key Terms 68
Exercises 69
References 71
3 A Framework For Steps and Errors In Web Surveys 73
3.1 Introduction 73
3.2 Theory 75
3.3 Application 88
3.4 Summary 89
Key Terms 90
Exercises 90
References 91
4 Sampling For Web Surveys 93
4.1 Introduction 93
4.2 Theory 95
4.2.1 Target Population 95
4.2.2 Sampling Frames 98
4.2.3 Basic Concepts of Sampling 103
4.2.4 Simple Random Sampling 106
4.2.5 Determining the Sample Size 109
4.2.6 Some Other Sampling Designs 112
4.2.7 Estimation Procedures 118
4.3 Application 123
4.4 Summary 128
Key Terms 129
Exercises 130
References 131
5 Errors In Web Surveys 133
5.1 Introduction 133
5.2 Theory 142
5.2.1 Measurement Errors 142
5.2.2 Nonresponse 164
5.3 Application 174
5.3.1 The Safety Monitor 174
5.3.2 Measurement Errors 175
5.3.3 Nonresponse 177
5.4 Summary 179
Key Terms 180
Exercises 182
References 185
6 Web Surveys and Other Modes of Data Collection 189
6.1 Introduction 189
6.1.1 Modes of Data Collection 189
6.1.2 The Choice of the Modes of Data Collection 190
6.2 Theory 194
6.2.1 Face-to-Face Surveys 194
6.2.2 Telephone Surveys 200
6.2.3 Mail Surveys 206
6.2.4 Web Surveys 211
6.2.5 Mobile Web Surveys 215
6.3 Application 222
6.4 Summary 230
Key Terms 231
Exercises 233
References 235
7 Designing A Web Survey Questionnaire 237
7.1 Introduction 237
7.2 Theory 240
7.2.1 The Road Map Toward a Web Questionnaire 240
7.2.2 The Language of Questions 249
7.2.3 Basic Concepts of Visualization 252
7.2.4 Answers Types (Response Format) 258
7.2.5 Web Questionnaires and Paradata 271
7.2.6 Trends in Web Questionnaire Design and Visualization 278
7.3 Application 281
7.4 Summary 282
Key Terms 283
Exercises 284
References 286
8 Adaptive and Responsive Design 291
8.1 Introduction 291
8.2 Theory 294
8.2.1 Terminology 294
8.2.2 Quality and Cost Functions 298
8.2.3 Strategy Allocation and Optimization 301
8.3 Application 309
8.4 Summary 316
Key Terms 316
Exercises 317
References 318
9 Mixed-Mode Surveys 321
9.1 Introduction 321
9.2 The Theory 326
9.2.1 What is Mixed-Mode? 326
9.2.2 Why Mixed-Mode? 334
9.3 Methodological Issues 343
9.3.1 Preventing Mode Effects Through Questionnaire Design 346
9.3.2 How to Mix Modes? 350
9.3.3 How to Compute Response Rates? 354
9.3.4 Avoiding and Adjusting Mode Effects for Inference 359
9.3.5 Mixed-Mode by Businesses and Households 370
9.4 Application 384
9.5 Summary 386
Key Terms 388
Exercises 388
References 390
10 The Problem of Under-Coverage 399
10.1 Introduction 399
10.2 Theory 405
10.2.1 The Internet Population 405
10.2.2 A Random Sample from the Internet Population 406
10.2.3 Reducing the Non-Coverage Bias 410
10.2.4 Mixed-Mode Data Collection 413
10.3 Application 414
10.4 Summary 417
Key Terms 418
Exercises 419
References 421
11 The Problem of Self-Selection 423
11.1 Introduction 423
11.2 Theory 431
11.2.1 Basic Sampling Theory 431
11.2.2 A Self-Selection Sample from the Internet Population 434
11.2.3 Reducing the Self-Selection Bias 439
11.3 Applications 444
11.3.1 Application 1: Simulating Self-Selection Polls 444
11.3.2 Application 2: Sunday Shopping in Alphen a/d Rijn 448
11.4 Summary 451
Key Terms 452
Exercises 453
References 455
12 Weighting Adjustment Techniques 457
12.1 Introduction 457
12.2 Theory 463
12.2.1 The Concept of Representativity 463
12.2.2 Post-Stratification 465
12.2.3 Generalized Regression Estimation 477
12.2.4 Raking Ratio Estimation 486
12.2.5 Calibration Estimation 490
12.2.6 Constraining the Values of Weights 491
12.2.7 Correction Using a Reference Survey 492
12.3 Application 500
12.4 Summary 506
Key Terms 508
Exercises 509
References 512
13 Use of Response Propensities 513
13.1 Introduction 513
13.2 Theory 517
13.2.1 A Simple Random Sample With Nonresponse 517
13.2.2 A Self-Selection Sample 520
13.2.3 The Response Propensity Definition 521
13.2.4 Models for Response Propensities 522
13.2.5 Correction Methods Based on Response Propensities 529
13.3 Application 535
13.3.1 Generation of the Population 536
13.3.2 Generation of Response Probabilities 537
13.3.3 Generation of the Sample 537
13.3.4 Computation of Response Propensities 537
13.3.5 Matching Response Propensities 537
13.3.6 Estimation of Population Characteristics 540
13.3.7 Evaluating the Results 541
13.3.8 Model Sensitivity 542
13.4 Summary 542
Key Terms 543
Exercises 544
References 546
14 Web Panels 549
14.1 Introduction 549
14.2 Theory 555
14.2.1 Under-Coverage 555
14.2.2 Recruitment 557
14.2.3 Nonresponse 563
14.2.4 Representativity 577
14.2.5 Weighting Adjustment for Panels 580
14.2.6 Panel Maintenance 582
14.3 Applications 585
14.3.1 Application 1: The Web Panel Pilot of Statistics Netherlands 585
14.3.2 Application 2: The U.K. Polling Disaster 589
14.4 Summary 592
Key Terms 593
Exercises 593
References 595
Index 599
Preface
The last 10 years have witnessed a significant increase in Internet penetration. What is particular about this growth is that a number of generations are currently experiencing the contemporary and highly technological environment. Social media, constant connectivity, and on-demand entertainments are innovations that Millennials (aged between 23 and 38 in 2019) adapted to as they grew up. For those born after 1996, the so-called Generation Z (aged between 7 and 22 in 2019), these innovations are mostly taken for granted, having been part of their lives from the beginning. The iPhone was launched in 2007, when the oldest members of Generation Z were 10. By the time they are in their teens, young Americans access the Internet mainly via mobile devices, Wi-Fi, and high-bandwidth cellular services. Pre-Millennial generations play an important role in the general population, but for them, this environment based on technological communication is a new experience.
The implications of some population subgroups having adapted to the technological environment (Millennials and pre-Millennials) while others have lived in this "always on" technological environment all their lives are of relevance for survey-based research, particularly in the case of web surveys. The way that questionnaires are administered undoubtedly has an impact which differs according to population group. Furthermore, the behavior of the respondents while participating depends on their digital experience, their generational characteristics, and their attitude toward technology in their lives. Therefore, surveys-and in particular web and mobile web surveys-have to adopt a number of changes in their methodology to take into account any differences in the cultural backgrounds of potential survey participants and the characteristics of the eventual devices used.
Due to high Internet penetration and the relatively low cost of conducting web surveys compared with other methods, the number of surveys being conducted via the Internet has increased dramatically over recent years. The panorama of survey-based research has changed drastically over the last few decades.
First there was a change from traditional paper-and-pencil interviewing (PAPI) to computer-assisted interviewing (CAI). Since the 1990s, there has been a gradual replacing of face-to-face surveys (CAPI), telephone surveys (CATI), and mail surveys (CASI, CSAQ) with web-based surveys. With the relatively recent diffusion of smartphones and other mobile devices, it has become possible to run mobile web surveys, i.e., questionnaires sent to interviewees may be submitted and also completed via mobile devices. A web survey is a simple way to access a large group of potential respondents. Questionnaires can be distributed at very low cost. They require no interviewers, and there are no mailing or printing costs involved. Surveys can be launched rapidly, and little time is lost between the moment the questionnaire is ready and the moment that fieldwork begins. Web surveys also offer interesting new possibilities, such as the use of multimedia (images, sound, animation, and video). Panel surveys are also moving toward data collection via the web.
The recent trend toward the use of big data and the integration of data sources will not render the role of web surveys obsolete, although they may in the future have a different role.
At first sight, web surveys appear to have much in common with other types of survey, seeming to be just another way to collect data, with questions asked over the Internet instead of face-to-face, by telephone or via e-mail. There are however a number of factors that may render the results of web surveys unreliable. Some examples are under-coverage, self-selection, and measurement errors. These can cause estimates of population characteristics to be biased, thus leading to incorrect conclusions being drawn from the data collected.
Under-coverage occurs when the target population is wider than the number of people with Internet access. This leads to bias in estimates in the case of relevant differences between those with Internet access and those without.
Self-selection is when a questionnaire is simply made available via Internet to all, with individuals nominating themselves.
A respondent is therefore anyone who happens to have Internet access, visits the website, and decides to take part in the survey. These participants generally differ significantly from nonparticipants.
General-population surveys that aim to provide reliable and accurate statistics are traditionally carried out face-to-face or by telephone. Interviewers are used to persuade people to take part and to help respondents to provide the right answers. In web surveys, there is no interviewer assistance, a fact that can have serious impact on the quality of the data collected.
The diffusion of smartphones increases the possibility for interviewees to be reached via their mobile device and to have the questionnaire completed via the same device, resulting in the current trend in running mobile web surveys. Consequently, there are new risks for error in the survey due to device characteristics and the behavior of the user.
The researcher should have in mind that when a web survey is run a mobile web survey takes place, if questionnaire is not blocked against mobile devices. Here, for simplicity the term web surveys is used, meaninig the mobile web survey is included. Summing up, web surveys afford several challanges and need reseacher be conscious of the methodological issues for a good survey. At the time beeing, collecting data through web surveys is going to become a common practice both in market research, academic research and official statistics. Knowledge about how to manage a web survey, risks, errors and advantages is important.
This book provides an insight into the possible use of web surveys and mobile web surveys for data collection. Web surveys allow for lower data collection costs. It is also expected that web surveys lead to increased response rates. Is this the case? What about the quality of the data collected? This book examines many theoretical and practical aspects of mobile web surveys and can therefore be considered as a handbook for those involved in practical survey research, including survey researchers working with official statistics (e.g., in national statistical institutes), academics, and commercial market research.
The book's two authors have widespread expertise in survey methodology. They come from two different countries (the Netherlands and Italy) and different research organizations (a national statistical institute and a university). They therefore provide a broad view on the various theoretical and practical aspects of mobile web surveying.
The second edition of the book involves a revision of each chapter with the following criteria:
(1) to maintain the existing text and content as much as possible, (2) to update the existing text and content with comments based on new literature and results, and (3) to add new paragraphs (if necessary) to cover new relevant topics (see the contents and chapter description below). A number of new examples have been provided, some of the existing examples have been updated or substituted, and some applications have been replaced. Updates have also been included to highlight new trends in web surveys and emerging solutions. There are two new chapters on topics concerning mobile web surveys: one presenting a flowchart to illustrate the steps involved in running a survey via web and the other examining adaptive design. It was therefore necessary to renumber the chapters in respect to the first edition.
The first two chapters of the book provide an introduction into web surveys. Chapter 1 provides a historic account of developments in survey research and shows how web surveys have become a tool for data collection. Section examines the Blaise system, which has been around for more than 30 years. New developments have taken place over the last 10 years, but no papers have been written on this subject. The section looks at the history and recent developments regarding Blaise; it was written by Lon Hofman and Mark Pierzchala and is published for the first time here.
Chapter 2 is an overview of basic aspects of web surveys. It describes how and where they can be used. Official statistics departments, research institutions, market research companies, and private forums are all interested in web surveys studying both households/individuals and businesses.
Chapter 3 presents a flowchart illustrating the steps (and sub-steps) of web surveys, each accompanied by a short description. The flowchart is of potential use to both practitioners as a guideline for how the survey process should be carried out and to researchers in highlighting and explaining the positioning of their studies at the different steps of the survey process. It is also useful in discussing the errors that can occur in different steps. The chapter provides an introduction to the framework and its structure and discusses the relevance of bearing in mind the framework and the survey steps when considering web survey errors. It then goes on to describe the concept of the step and the structure of the flowchart, breaking down the web survey process into six main steps. These are analyzed in detail, and an overview of survey errors is provided.
Chapter 4 examines the aspects of sampling. It is stressed that valid population inference is possible only if some...
