Register-based Statistics
Description
Rediscover this authoritative guide to register-based statistics filled with significant new improvements
In the newly revised Third Edition of Register-based Statistics: Registers and the National Statistical System, Anders Wallgren and Britt Wallgren deliver a robust exploration of how register-based statistics can be used to its fullest potential. The authors describe how statistical institutes can work on long-term projects to improve administrative systems, as well as estimation methods that can improve the quality of statistical estimates based on registers with quality problems. Readers will also discover how to improve the ways register-statistical issues are introduced, as well as how to create population registers.
Finally, the authors draw on their experience from teaching and consulting in several countries to explain how to implement register-based statistics.
Key features of the third edition:
* Discusses the problems new register countries face
* Explains how registers will improve the efficiency of the national statistical system
* Clarifies the importance of the system approach
* Describes how a statistical population register can be created
* Registers-based statistics require new skills and understanding of new concepts
* Many important quality indicators are described
* Explains difficult topics in a pedagogic way
Perfect for staff at national statistical institutes and administrative and ministerial authorities belonging to national statistical systems, Register-based Statistics will also prove to be an indispensable resource for undergraduate and graduate students in statistics programs and courses, as well as survey researchers and practitioners.
More details
Other editions
Additional editions
Persons
Anders Wallgren and Britt Wallgren worked for many years in teaching and research as senior lecturers in statistics at Stockholm, Linköping and Örebro universities. They have also worked as consultants for several Swedish manufacturing industries. In connection with these consulting assignments, they began working with corporate administrative registers and developed statistical methods for business intelligence.
After their academic career, they came to Statistics Sweden and worked for about 20 years at the Department of Research and Development. They worked mainly with survey methodology for the statistical registers which form the basis for the production of official statistics in Sweden.
Since 2011 they have provided courses and consulting services on behalf of the Inter-American Development Bank in several Latin American countries and in the Caribbean.
Content
Index to Charts xi
Preface xix
Chapter 1 Censuses, Sample Surveys and Register Surveys 1
1.1 The national statistical system 2
1.2 The traditional census-based system 3
1.3 New sources: Administrative registers and Big data 5
1.4 Basic concepts and terms 7
1.4.1 What is a register? 7
1.4.2 Databases, records and observations 8
1.4.3 What is a register survey? 10
1.4.4 A register survey: The Income and Taxation Register 12
1.5 New demands and opportunities require new methods 14
1.5.1 A new paradigm is necessary 14
1.5.2 New statistical methods 15
1.5.3 The basic principles of register-based statistics 18
1.6 Preconditions for register-based statistics 19
1.6.1 Reliable administrative systems 20
1.6.2 Legal base and public approval 21
1.6.3 Political support to strengthen the statistical system 23
Chapter 2 The Transition to a Register-based Production System 25
2.1 First obstacle: How to gain access to microdata? 26
2.2 Protection of privacy and confidentiality 26
2.3 Second obstacle: How to take care of dirty data? 29
2.4 The new production process 30
2.4.1 Contacts with administrative authorities 31
2.4.2 Metadata have a new role 31
2.4.3 Anonymisation of identity numbers 32
2.4.4 Editing of a single administrative register 33
2.4.5 Organising the work with administrative registers 36
2.5 Third obstacle: The national registration system 37
2.5.1 Legislation governs access to data 38
2.5.2 Too many registers, but no good registers - what to do? 38
2.5.3 Legislation rules obligations to report and what to report 40
2.6 Why has the census been so important? 41
2.7 Creating the register system 42
2.7.1 Where do you live? 43
2.7.2 Where do you work? 45
2.7.3 With whom do you live? 46
2.7.4 A centralised or decentralised national system? 47
2.8 Register surveys and estimation methods 48
2.9 A traditional census or a register-based census? 49
Chapter 3 The Nature of Administrative Data 51
3.1 Comparing questionnaire and register data 51
3.1.1 A questionnaire to persons compared with register data 51
3.1.2 An enterprise questionnaire compared with register data 54
3.2 Enterprise registers for combined use 56
3.2.1 Corrections in accounting data 57
3.2.2 Missing values in accounting data 58
3.2.3 Administrative and statistical information systems 59
3.3 Measurement errors in questionnaire and register data 60
3.3.1 Measurement errors 61
3.3.2 Taxation errors 62
Chapter 4 Building the System - Record Linkage 65
4.1 Record linkage 65
4.2 Record linkage in the Nordic countries 66
4.3 Deterministic record linkage 68
4.4 Creating variables by adjoining and aggregation 70
4.5 Probabilistic record linkage 73
4.6 Four causes of matching errors 79
4.7 The statistical system and record linkage 82
Chapter 5 Building the System - Quality Assessment 85
5.1 Four quality concepts 85
5.2 Making an inventory of potential sources 87
5.3 How can a source be used? 87
5.4 Quality assessment in a register-based production system 90
5.4.1 Analysing metadata 91
5.4.2 Analysis and data editing of the source 92
5.4.3 Comparing a source with the base register 92
5.4.4 Comparing a source with surveys with similar variables 93
5.5 Output data quality and quality of estimates 94
5.5.1 Analysing quality with a test census 94
5.5.2 Analysing quality with samples from the new register 95
5.5.3 Analysing quality with area samples 96
5.5.4 Measuring quality of basic register variables with the LFS 98
5.6 A coordinated system of registers 98
5.6.1 Are the base registers a coordinated system? 98
5.6.2 Quality indicators at the system level 99
5.7 Using the quality indicators 101
Chapter 6 Building the System - Editing Register Data 107
6.1 Editing in register surveys 108
6.2 Editing of a single administrative register 109
6.3 Consistency editing 110
6.3.1 Consistency editing - is the population correct? 111
6.3.2 Consistency editing - are the units correct? 118
6.3.3 Consistency editing - are the variables correct? 120
6.4 Case studies - editing register data 121
6.4.1 Editing work within the Income and Taxation Register 121
6.4.2 Editing work within the Income Statement Register 123
6.4.3 What more can be learned from these examples? 124
6.5 Editing, quality assessment and survey design 125
6.5.1 Survey design in a register-based production system 125
6.5.2 Survey design - management problems 127
6.5.3 Total survey error in a register-based system 128
Chapter 7 Building the System - The Population Register 129
7.1 Inventory of sources 131
7.1.1 Time references 131
7.1.2 Activities or 'signs of life' 131
7.2 The Population Register based on full information 133
7.2.1 Object types - Changing and unchanging registers 133
7.2.2 Variables with different functions in the system 134
7.2.3 Updating the Population Register 136
7.2.4 Registers and time 137
7.2.5 Variables and time 140
7.3 The Population Register in new register countries 140
7.3.1 Different systems of identity numbers 141
7.3.2 Problems in countries without a central Population Register 142
7.3.3 How to improve coverage of the Population Register 143
7.3.4 Inventory of sources - addresses and time references 146
7.4 Methods to measure and improve quality 148
7.4.1 Three kinds of surveys should be combined 148
7.4.2 A new register-based system for statistics on persons 150
7.5 Conclusions 151
7.6 Challenges in old register countries 152
Chapter 8 The Population Register - Estimation Methods 155
8.1 Estimation in sample surveys and register surveys 156
8.1.1 Estimation methods for register surveys that use weights 157
8.1.2 Calibration of weights in register surveys 157
8.2 Calibration of weights - the Swedish LFS 161
8.2.1 Use of auxiliary information in the LFS 161
8.2.2 Nonresponse bias in the LFS 162
8.3 Calibration - where do people live? 163
8.4 Methods to handle overcoverage 167
Chapter 9 Defining Register Populations - Coverage Errors 171
9.1 Defining a register's object set 172
9.1.1 Defining a population 172
9.1.2 Can you alter data from the National Tax Agency? 176
9.1.3 Defining a population - the Farm Register 176
9.1.4 Defining a population - integrated registers 178
9.2 Defining a calendar year population 179
9.2.1 Defining a population - frame or register population? 180
9.2.2 Sampling paradigm versus register paradigm 184
Chapter 10 Building the System - The Business Register 185
10.1 The Business Register and the National Accounts 185
10.2 The base register for economic statistics 187
10.3 The scope of the register and choice of object types 188
10.3.1 The register with legal units and local units 189
10.3.2 The register with enterprise units and kind of activity units 191
10.4 Inventory of sources 195
10.5 Creating and maintaining the Business Register 198
Chapter 11 The Business Register - Estimation Methods 201
11.1 Multi-valued variables 202
11.2 Estimation methods 205
11.2.1 Occupation in the Activity and Occupation Registers 206
11.2.2 Industrial classification in the Business Register 210
11.2.3 Estimates from different register versions 213
11.3 Application of the method 214
11.3.1 Change of industry and time series quality 215
11.3.2 Transformation of weights 217
11.4 A decentralised or centralised statistical system? 218
11.4.1 The Calendar Year Register and the National Accounts 219
11.4.2 Choosing the best source for the National Accounts 220
11.5 Conclusions 224
Chapter 12 Censuses, Sample Surveys and Register Surveys -Conclusions 227
12.1 Attitudes towards the register-based census 227
12.2 The new national statistical system 231
12.2.1 The system of base registers 232
12.2.2 Activity registers and longitudinal registers 234
12.3 Survey design 237
12.3.1 Sample survey design 237
12.3.2 Register survey design 238
12.3.3 Creating register variables 241
12.4 Survey quality 245
12.4.1 Quality of registers and register surveys 246
12.4.2 The integration process - integration errors 247
12.4.3 Frame errors 247
12.5 Organising the new production system 248
12.5.1 Enterprise architecture and the register system 248
12.5.2 The register system and data warehousing 249
12.5.3 Missing values - a system-based approach 252
12.6 Final remarks 254
12.6.1 The Statistical Population Register 254
12.6.2 The system of base registers 255
References 257
Index 261
Index to Charts
Chapter 1 Censuses, Sample Surveys and Register Surveys
Chart 1.1 The Statistical System in a country can consist of the following actors
Chart 1.2 The Statistical System in a country can consist of the following surveys
Chart 1.3 Employment with census data
Chart 1.4 Employment with register data
Chart 1.5 The year of establishing new statistical registers in the Nordic countries
Chart 1.6 Example of a register and data matrix
Chart 1.7 A conceptual database model with three database tables
Chart 1.8 A database on individuals with three database tables
Chart 1.9 Two data matrices for different statistical purposes
Chart 1.10 From administrative registers to statistical registers - overview
Chart 1.11 From administrative registers to statistical registers - register processing
Chart 1.12 Different data sources for the Income and Taxation Register (I &T)
Chart 1.13 Employees by economic activity, November 2004, thousands
Chart 1.14 Register-based census statistics for one small municipality in Sweden
Chart 1.15 Four principles for using administrative registers for statistics
Chart 1.16 Two preconditions for using administrative registers for statistics
Chapter 2 The Transition to a Register-based Production System
Chart 2.1 The transition entails a fundamental change of statistical methods
Chart 2.2 An administrative register - unprocessed data in the input database
Chart 2.3 Corresponding statistical register - processed data in throughput database
Chart 2.4 Transforming administrative registers into statistical registers
Chart 2.5 Metadata system for input data from administrative registers
Chart 2.7 Errors in profit and loss statements from limited companies, SEK million
Chart 2.8 Profit and loss statements for limited companies, SEK million
Chart 2.9 Registration of persons & demographic events in a Latin American country
Chart 2.10 Uncoordinated work with registers
Chart 2.11 Cooperation regarding the central population register
Chart 2.12 Sweden's system 1967-
Chart 2.13 Sweden's system 1985-
Chart 2.15 Survey design: reduce costs and/or improve quality
Chapter 3 The Nature of Administrative Data
Chart 3.1 Yearly turnover for the same enterprises in three sources, USD million
Chart 3.2 Distance for each ordered observation
Chart 3.3 Complete groups of enterprises
Chart 3.4 Three registers, Invoice Register, Client Register, Item Register
Chart 3.5 Statistical Sales Register for January 2012 - four transactions
Chart 3.6 Administrative Invoice Register
Chart 3.7 Statistical Invoice Register
Chart 3.8 Administrative Item Register
Chart 3.9 Statistical Item Register
Chart 3.10 Measurement errors - comparison of data collection methods
Chapter 4 Building the System - Record Linkage
Chart 4.1a Matching without errors in the matching key (PIN)
Chart 4.1b Matching with errors in the matching key (PIN)
Chart 4.1c Matching with errors in the matching key (First name and Surname)
Chart 4.1d Matching without errors in the matching key (PIN)
Chart 4.2a The relations between persons, activities and establishments
Chart 4.2b Wage sums for persons and establishments created by aggregation
Chart 4.2c Industry and sex as derived variables for jobs created by adjoining
Chart 4.2d Industry and number of employees as derived variables
Chart 4.3 Spelling errors, spelling variations and unstandardized addresses
Chart 4.4 Two ways of searching for duplicates in a Birth Register
Chart 4.5 Calculation of probabilities for outcomes for 1st name, 1st and 2nd surname
Chart 4.6 The eight outcomes for the matching key: 1st name, 1st and 2nd surname
Chart 4.7 Some outcomes for the matching key: Name and date of birth of the mother
Chart 4.8 Yearly turnover for the same legal units in three sources, USD million
Chart 4.9 Comparing gross yearly pay in quarterly and annual registers
Chapter 5 Building the System - Quality Assessment
Chart 5.1 A register survey: From administrative registers to statistical estimates
Chart 5.2 Statistical registers that are suitable for early development
Chart 5.3 Input and output data and the production process
Chart 5.4 The work with quality assessment of an administrative source
Chart 5.5 Indicators A1-A9 of input data quality - Relevance
Chart 5.6 Indicators B1-B7 of input data quality - Accuracy
Chart 5.7 Indicators C1-C5 of accuracy when comparing with the base register
Chart 5.8 Indicators D1-D4 of accuracy when comparing with related surveys
Chart 5.9 Coverage errors in the population register for Galapagos 2015
Chart 5.10 Coverage errors and area sample estimates of the population by district
Chart 5.11 Creating registers by linking different kinds of statistical units
Chart 5.12 Indicators E1-E4 of the quality of base registers in the system
Chart 5.13 Information from the administrative authority - relevance
Chart 5.14 Information from analysis and data editing of the source - accuracy
Chart 5.15 Information from integrating the source with base registers - accuracy
Chart 5.16 Information from integrating the source with related surveys - accuracy
Chapter 6 Building the System - Editing Register Data
Chart 6.1 Editing in sample surveys and register surveys
Chart 6.2 Automatic editing and imputation. Tax returns, 464 567 small enterprises
Chart 6.3a The Business Register and the Annual Pay Register
Chart 6.3b After matching the Business Register and the Annual Pay Register
Chart 6.3c The statistical Annual Pay Register
Chart 6.4a Combining the Business, Annual and Quarterly Pay Registers
Chart 6.4b Final version of the statistical Annual Pay Register
Chart 6.5 All relevant sources should be considered simultaneously
Chart 6.6 Comparing gross annual pay in a quarterly and yearly source, microdata
Chart 6.7 The unit problem in business data
Chart 6.8 Comparing gross annual pay in QGP and AGP, microdata
Chart 6.9 Comparing gross annual pay in QGP and AGP, macrodata
Chart 6.10 The system of registers and surveys that was analysed
Chapter 7 Building the System - The Population Register
Chart 7.1 Decentralised but coordinated process to create registers on persons
Chart 7.2 The system of registers that constitutes the Population Register at the NSO
Chart 7.3 Different types of variables in the Population Register
Chart 7.4a The Population Register at December 31, 20121
Chart 7.4b Notifications regarding demographic events delivered 1 February
Chart 7.4c Old register matched with the new notifications
Chart 7.4d Updated register 1 February
Chart 7.5 Calendar year register for
Chart 7.6 Events register for 2012 regarding change of address
Chart 7.7 Historical register regarding change of address
Chart 7.8 Longitudinal register for 2010-
Chart 7.9 Many registers, but no register that covers the entire population
Chart 7.10 The first steps towards a new statistical system
Chart 7.11 Different parts of the target population and the desired variables
Chart 7.12 Estimation of undercoverage
Chart 7.13 Coverage errors and area sample estimates of the population by district
Chart 7.14 Coverage errors and register-based sample estimates of population
Chart 7.15 A register survey: From administrative registers to statistical estimates
