Direction Dependence in Statistical Modeling
Description
Direction Dependence in Statistical Modeling: Methods of Analysis incorporates the latest research for the statistical analysis of hypotheses that are compatible with the causal direction of dependence of variable relations. Having particular application in the fields of neuroscience, clinical psychology, developmental psychology, educational psychology, and epidemiology, direction dependence methods have attracted growing attention due to their potential to help decide which of two competing statistical models is more likely to reflect the correct causal flow.
The book covers several topics in-depth, including:
* A demonstration of the importance of methods for the analysis of direction dependence hypotheses
* A presentation of the development of methods for direction dependence analysis together with recent novel, unpublished software implementations
* A review of methods of direction dependence following the copula-based tradition of Sungur and Kim
* A presentation of extensions of direction dependence methods to the domain of categorical data
* An overview of algorithms for causal structure learning
The book's fourteen chapters include a discussion of the use of custom dialogs and macros in SPSS to make direction dependence analysis accessible to empirical researchers.
More details
Other editions
Additional editions
Persons
WOLFGANG WIEDERMANN is Associate Professor at the University of Missouri-Columbia. He received his Ph.D. in Quantitative Psychology from the University of Klagenfurt, Austria. His primary research interests include the development of methods for causal inference, methods to determine the causal direction of dependence in observational data, and methods for person-oriented research settings. He has edited books on advances in statistical methods for causal inference (with von Eye, Wiley) and new developments in statistical methods for dependent data analysis in the social and behavioral sciences (with Stemmler and von Eye).
DAEYOUNG KIM is Associate Professor of Mathematics and Statistics at the University of Massachusetts, Amherst. He received his Ph.D. from the Pennsylvania State University in Statistics. His original research interests were in likelihood inference in finite mixture modelling including empirical identifiability and multimodality, development of geometric and computational methods to delineate multidimensional inference functions, and likelihood inference in incompletely observed categorical data, followed by a focus on the analysis of asymmetric association in multivariate data using (sub)copula regression.
ENGIN A. SUNGUR has a B.A. in City and Regional Planning (Middle East Technical University, METU, Turkey), M.S. in Applied Statistics, METU, M.S. in Statistics (Carnegie-Mellon University, CMU) and Ph.D. in Statistics (CMU). He taught at Carnegie-Mellon University, University of Pittsburg, Middle East Technical University, and University of Iowa. Currently, he is a Morse-Alumni distinguished professor of statistics at University of Minnesota Morris. He is teaching statistics for more than 38 years, 29 years of which is at the University of Minnesota Morris. His research areas are dependence modeling with emphasis on directional dependence, modern multivariate statistics, extreme value theory, and statistical education.
ALEXANDER VON EYE is Professor Emeritus of Psychology at Michigan State University (MSU). He received his Ph.D. in Psychology from the University of Trier, Germany. He received his accreditation as Professional Statistician from the American Statistical Association (PSTATTM). His research focuses (1) on the development and testing of statistical methods for the analysis of categorical and longitudinal data, and for the analysis of direction dependence hypotheses. In addition (2), he is member of a research team at MSU (with Bogat, Levendosky, and Lonstein) that investigates the effects of violence on women and their newborn children. His third area of interest (3) concerns theoretical developments and applied analysis of person-orientation in empirical research.
Content
About the Editors xv
Notes on Contributors xvii
Acknowledgments xxi
Preface xxiii
Part I Fundamental Concepts of Direction Dependence 1
1 From Correlation to Direction Dependence Analysis 1888-2018 3
Yadolah Dodge and Valentin Rousson
1.1 Introduction 3
1.2 Correlation as a Symmetrical Concept of X and Y 4
1.3 Correlation as an Asymmetrical Concept of X and Y 5
1.4 Outlook and Conclusions 6
References 6
2 Direction Dependence Analysis: Statistical Foundations and Applications 9
Wolfgang Wiedermann, Xintong Li, and Alexander von Eye
2.1 Some Origins of Direction Dependence Research 11
2.2 Causation and Asymmetry of Dependence 13
2.3 Foundations of Direction Dependence 14
2.3.1 Data Requirements 15
2.3.2 DDA Component I: Distributional Properties of Observed Variables 16
2.3.3 DDA Component II: Distributional Properties of Errors 19
2.3.4 DDA Component III: Independence Properties 20
2.3.5 Presence of Confounding 21
2.3.6 An Integrated Framework 24
2.4 Direction Dependence in Mediation 29
2.5 Direction Dependence in Moderation 32
2.6 Some Applications and Software Implementations 34
2.7 Conclusions and Future Directions 36
References 38
3 The Use of Copulas for Directional Dependence Modeling 47
Engin A. Sungur
3.1 Introduction and Definitions 47
3.1.1 Why Copulas? 48
3.1.2 Defining Directional Dependence 48
3.2 Directional Dependence Between Two Numerical Variables 51
3.2.1 Asymmetric Copulas 52
3.2.2 Regression Setting 59
3.2.3 An Alternative Approach to Directional Dependence 62
3.3 Directional Association Between Two Categorical Variables 70
3.4 Concluding Remarks and Future Directions 74
References 75
Part II Direction Dependence in Continuous Variables 79
4 Asymmetry Properties of the Partial Correlation Coefficient: Foundations for Covariate Adjustment in Distribution-Based Direction Dependence Analysis 81
Wolfgang Wiedermann
4.1 Asymmetry Properties of the Partial Correlation Coefficient 84
4.2 Direction Dependence Measures when Errors Are Non-Normal 86
4.3 Statistical Inference on Direction Dependence 89
4.4 Monte-Carlo Simulations 90
4.4.1 Study I: Parameter Recovery 90
4.4.1.1 Results 91
4.4.2 Study II: CI Coverage and Statistical Power 91
4.4.2.1 Type I Error Coverage 94
4.4.2.2 Statistical Power 94
4.5 Data Example 98
4.6 Discussion 101
4.6.1 Relation to Causal Inference Methods 103
References 105
5 Recent Advances in Semi-Parametric Methods for Causal Discovery 111
Shohei Shimizu and Patrick Blöbaum
5.1 Introduction 111
5.2 Linear Non-Gaussian Methods 113
5.2.1 LiNGAM 113
5.2.2 Hidden Common Causes 115
5.2.3 Time Series 118
5.2.4 Multiple Data Sets 119
5.2.5 Other Methodological Issues 119
5.3 Nonlinear Bivariate Methods 119
5.3.1 Additive Noise Models 120
5.3.1.1 Post-Nonlinear Models 121
5.3.1.2 Discrete Additive Noise Models 121
5.3.2 Independence of Mechanism and Input 121
5.3.2.1 Information-Geometric Approach for Causal Inference 122
5.3.2.2 Causal Inference with Unsupervised Inverse Regression 123
5.3.2.3 Approximation of Kolmogorov Complexities via the Minimum Description Length Principle 123
5.3.2.4 Regression Error Based Causal Inference 124
5.3.3 Applications to Multivariate Cases 125
5.4 Conclusion 125
References 126
6 Assumption Checking for Directional Causality Analyses 131
Phillip K. Wood
6.1 Epistemic Causality 135
6.1.1 Example Data Set 136
6.2 Assessment of Functional Form: Loess Regression 137
6.3 Influential and Outlying Observations 140
6.4 Directional Dependence Based on All Available Data 141
6.4.1 Studentized Deleted Residuals 143
6.4.2 Lever 143
6.4.3 DFFITS 144
6.4.4 DFBETA 145
6.4.5 Results from Influence Diagnostics 145
6.4.6 Directional Dependence Based on Factor Scores 148
6.5 Directional Dependence Based on Latent Difference Scores 149
6.6 Direction Dependence Based on State-Trait Models 153
6.7 Discussion 156
References 163
7 Complete Dependence: A Survey 167
Santi Tasena
7.1 Basic Properties 168
7.2 Measure of Complete Dependence 171
7.3 Example Calculation 177
7.4 Future Works and Open Problems 180
References 181
Part III Direction Dependence in Categorical Variables 183
8 Locating Direction Dependence Using Log-Linear Modeling, Configural Frequency Analysis, and Prediction Analysis 185
Alexander von Eye and Wolfgang Wiedermann
8.1 Specifying Directional Hypotheses in Categorical Variables 187
8.2 Types of Directional Hypotheses 192
8.2.1 Multiple Premises and Outcomes 192
8.3 Analyzing Event-Based Directional Hypotheses 193
8.3.1 Log-Linear Models of Direction Dependence 193
8.3.1.1 Identification Issues 197
8.3.2 Confirmatory Configural Frequency Analysis (CFA) of Direction Dependence 198
8.3.3 Prediction Analysis of Cross-Classifications 200
8.3.3.1 Descriptive Measures of Prediction Success 202
8.4 Data Example 203
8.4.1 Log-Linear Analysis 205
8.4.2 Configural Analysis 206
8.4.3 Prediction Analysis 208
8.5 Reversing Direction of Effect 209
8.5.1 Log-Linear Modeling of the Re-Specified Hypotheses 209
8.5.2 CFA of the Re-Specified Hypotheses 210
8.5.3 PA of the Re-Specified Hypotheses 212
8.6 Discussion 212
References 215
9 Recent Developments on Asymmetric Association Measures for Contingency Tables 219
Xiaonan Zhu, Zheng Wei, and Tonghui Wang
9.1 Introduction 219
9.2 Measures on Two-Way Contingency Tables 220
9.2.1 Functional Chi-Square Statistic 220
9.2.2 Measures of Complete Dependence 222
9.2.3 A Measure of Asymmetric Association Using Subcopula-Based Regression 223
9.3 Asymmetric Measures of Three-Way Contingency Tables 225
9.3.1 Measures of Complete Dependence for Three Way Contingency Table 225
9.3.2 Subcopula Based Measure for Three Way Contingency Table 232
9.3.3 Estimation 235
9.4 Simulation of Three-Way Contingency Tables 237
9.5 Real Data of Three-Way Contingency Tables 239
References 240
10 Analysis of Asymmetric Dependence for Three-Way Contingency Tables Using the Subcopula Approach 243
Daeyoung Kim and Zheng Wei
10.1 Introduction 243
10.2 Review on Subcopula Based Asymmetric Association Measure for Ordinal Two-Way Contingency Table 245
10.3 Measure of Asymmetric Association for Ordinal Three-Way Contingency Tables via Subcopula Regression 248
10.3.1 Subcopula Regression-Based Asymmetric Association Measures 248
10.3.2 Estimation 251
10.4 Numerical Examples 253
10.4.1 Sensitivity Analysis 253
10.4.2 Data Analysis 257
10.5 Conclusion 260
10.A Appendix 261
10.A.1 The Proof of Proposition 10.1 261
References 262
Part IV Applications and Software 265
11 Distribution-Based Causal Inference: A Review and Practical Guidance for Epidemiologists 267
Tom Rosenström and Regina García-Velázquez
11.1 Introduction 267
11.2 Direction of Dependence in Linear Regression 268
11.3 Previous Epidemiologic Applications of Distribution-Based Causal Inference 271
11.4 A Running Example: Re-Visiting the Case of Sleep Problems and Depression 273
11.5 Evaluating the Assumptions in Practical Work 274
11.5.1 Testing Linearity 275
11.5.2 Testing Non-Normality 276
11.5.3 Testing Independence 277
11.6 Distribution-Based Causality Estimates for the Running Example 278
11.7 Conducting Sensitivity Analyses 279
11.7.1 Convergent Evidence from Multiple Estimators 279
11.7.2 Simulation-Based Analysis of Robustness to Latent Confounding 279
11.7.2.1 Obtain Data-Based Parameters 281
11.7.2.2 Defining Parameters and Simulation Conditions 281
11.7.2.3 Defining the Simulation Model 282
11.7.2.4 Run Simulation and Interpret Results 283
11.8 Simulation-Based Analysis of Statistical Power 284
11.9 Triangulating Causal Inferences 288
11.10 Conclusion 291
References 292
12 Determining Causality in Relation to Early Risk Factors for ADHD: The Case of Breastfeeding Duration 295
Joel T. Nigg, Diane D. Stadler, Alexander von Eye, and Wolfgang Wiedermann
12.1 Method 298
12.1.1 Participants 298
12.1.1.1 Recruitment and Identification 298
12.1.1.2 Parental Psychopathology 299
12.1.1.3 Ethical Standards 300
12.1.2 Exclusion Criteria 300
12.1.2.1 Assessment of Breastfeeding Duration 300
12.1.3 Covariates 301
12.1.3.1 Parental Education 301
12.1.3.2 Primary Residence and Family Income 301
12.1.3.3 Parental Occupational Status 301
12.1.4 Data Reduction and Data Analysis 301
12.1.4.1 Parental ADHD 301
12.1.4.2 Data Reduction 301
12.1.4.3 Data Analysis 302
12.2 Results 304
12.2.1 Study Participant Demographic and Clinical Characteristics 304
12.3 Discussion 316
12.3.1 Limitations 317
12.3.2 Question of Causality 317
Acknowledgments 318
References 318
13 Direction of Effect Between Intimate Partner Violence and Mood Lability: A Granger Causality Model 325
G. Anne Bogat, Alytia A. Levendosky, Jade E. Kobayashi, and Alexander von Eye
13.1 Introduction 325
13.1.1 Definitions and Frequency of IPV 326
13.1.2 Depression, Mood and IPV 329
13.1.2.1 Depression and IPV 329
13.1.2.2 Mood and IPV 330
13.1.3 Summary 332
13.2 Methods 333
13.2.1 Participants 333
13.2.2 Measures 333
13.2.2.1 Daily Diary Questions 333
13.2.3 Procedures 334
13.3 Results 334
13.3.1 Data Consolidation 334
13.3.2 Descriptive Statistics 335
13.3.3 Model Development 335
13.3.4 Granger Causality Analyses 337
13.4 Discussion 341
References 343
14 On the Causal Relation of Academic Achievement and Intrinsic Motivation: An Application of Direction Dependence Analysis Using SPSS Custom Dialogs 351
Xintong Li and Wolfgang Wiedermann
14.1 Direction of Dependence in Linear Regression 352
14.1.1 Distributional Properties of x and y 353
14.1.2 Distributional Properties of ex and ey 354
14.1.3 Independence of Error Terms with Predictor Variable 355
14.1.4 DDA in Confounded Models 356
14.1.5 DDA in Multiple Linear Regression Models 356
14.2 The Causal Relation of Intrinsic Motivation and Academic Achievement 359
14.2.1 High School Longitudinal Study 2009 360
14.3 Direction Dependence Analysis Using SPSS 363
14.3.1 Variable Distributions and Assumption Checks 363
14.3.2 Residual Distributions 366
14.3.3 Independence Properties 368
14.3.4 Summary of DDA Results 369
14.4 Conclusions 371
14.4.1 Extensions and Future Work 372
References 372
Author Index 379
Subject Index 395
Preface
Questions concerning causation are omnipresent in the empirical sciences. In non-experimental research, however, it is often hard to determine the status of variables as cause and effect. Temporal order alone is of limited use, unless one observes antecedents and the beginning of a chain of events. That is, even when a putative explanatory variable (x) is measured earlier in time than the (putative) outcome (y), one cannot rule out that an outcome, measured at an earlier point in time, may have caused x. Similarly, temporality alone does not prevent causal effect estimates from being biased unless one is able to adjust for all relevant (potentially time-varying) confounders (Bellemare, Masaki, & Pepinsky, 2017). Cross-sectional research has often been looked-down upon because it is deemed of little use for the analysis of hypotheses that are compatible with (possibly competing) theories of causality. Based on cross-sectional data alone, for example, one is not able to distinguish whether a relation between x and y is observed because of an underlying causal model of the form x y (i.e. x causes y), the reverse-causal model y x (y causes x), or whether the observed relation is spurious due to (total or partial) confounding, x u y.
Limitations of longitudinal and cross-sectional observational research are (partly) rooted in the limitations of the statistical methods that are routinely applied to analyze dependence structures. In both research designs, covariance-based methods (such as correlational, linear regression, and structural equation modeling techniques) are de rigueur. Although, these methods can be useful in the estimation of the magnitude of causal effects (provided that certain unconfoundedness conditions are fulfilled, see, e.g. Pearl, 2009), they do not help to empirically distinguish between cause and effect. For example, in the standardized case, linear regression parameters for the model x y are identical to the ones that are estimated for the reverse regression, y x (von Eye & DeShon, 2012). These symmetry properties of the linear regression model have been known since its early origins (Galton, 1886). In fact, the observation that regression is inherently symmetric was one of the reasons why Francis Galton (the "founding father" of linear regression) changed his characterization of the phenomenon that previously suppressed hereditary traits can re-appear from a phenomenon of reversion to a phenomenon of regression (Gorroochurn, 2016). In other words, symmetry properties influenced how linear regression was conceptualized as a statistical tool. Similarly, symmetry properties of conventional representations of the Pearson product-moment correlation (for an overview of various facets of the Pearson correlation see, for example, Rodgers and Nicewander (1988), Rovine and von Eye (1997), Falk and Well (1997), and Nelsen (1998)) certainly contributed to the widespread and well-known mantra that correlation does not imply causation and to the belief that the means of statistic cannot be used to establish the causal direction of dependence.
Fortunately, this state of affairs has changed recently. It did take statisticians until the beginning of the new millennium to get a handle on the issue of direction dependence. But in 2000, Dodge and Rousson derived, within the framework of the linear regression model, the relation between cause and effect variables, for the (not so) particular case in which the cause variable is asymmetrically distributed. Specifically, these authors showed that variable information beyond means, variances, and covariances (e.g. skewness and co-skewness) can be used to empirically determine which of two variables, is more likely to be the cause and which is more likely to be the effect. Focusing on asymmetry properties of the linear regression and the Pearson correlation, the work by Dodge and Rousson (2000) initiated a new topic and line of statistical research, that of the development and application of methods for the analysis of direction dependence and causal hypotheses. Dodge and Rousson (2000) focused on asymmetry that emerges from marginal variable distributions. Asymmetry properties based on error distributions have later been proposed by Wiedermann, Hagmann, and von Eye (2015), Wiedermann and von Eye (2015b), and Wiedermann and Hagmann (2016). Extensions to measurement error models were recently discussed in Wiedermann, Merkle, and von Eye (2018). The second seminal paper in this new line of research was published in 2005 by Engin A. Sungur (see Sungur (2005a); a discussion of copulas in the regression context is given by Sungur (2005b)). While Dodge and Rousson's (2000) initial work focused on determining the direction of dependence through studying the marginal behavior of distributions, Sungur (2005a) proposed to study the behavior of joint variable distributions by making use of copulas. This copula-based direction dependence approach constitutes a second line of research that allows researchers to analyze cause-effect properties of variables while accounting for potential differences in marginal distributions. Copula-based directional dependence analysis has experienced rapid development. Various extension have been proposed by, e.g. Kim and Kim (2014, 2016), Wei and Kim (2017, 2018), and Kim and Hwang (2019) - more recent applications of the approach are given by Lee and Kim (2019) and Kim, Lee and Xiao (2019). The third seminal paper in the development of methods to distinguish between cause and effect variables was published by Shimizu and colleagues in 2006 proposing the linear non-Gaussian acyclic model (LiNGAM) - a causal machine learning algorithm for non-normal variables that is closely related to independent component analysis (Hyvärinen, Karhunen, & Oja, 2001). LiNGAM rapidly developed in the area of machine learning research and has been extended to nonlinear variable relations (Zhang & Hyvärinen, 2016), models with hidden common causes (Hoyer, Shimizu, Kerminen, & Palviainen, 2008; Shimizu & Bollen, 2014), and mixed (continuous and categorical) data (Yamayoshi, Tsuchida, & Yadohisa, 2020), to name a few. For an overview of recent advances in causal machine learning, see Guyon, Statnikov, and Batu (2019).
The present book is concerned with novel statistical approaches to the analysis of the causal direction of dependence of variables in both, exploratory (i.e. learning the causal structures from observational data without background knowledge) and confirmatory (i.e. testing a priori existing competing causal theories) research scenarios, and presents original work in four modules. In the first module, Fundamental Concepts of Direction Dependence, Dodge and Rousson (Chapter 1) introduce the well-known Pearson correlation coefficient as an asymmetric concept of two variables which (as discussed above) served as a starting point for several lines of direction dependence research. Further, the authors provide a reminder that working with non-normality of variables (as a key requirement to derive asymmetry properties in the linear case) bears challenges in practice (e.g. distinguishing between non-normality as a characteristic of the construct under study versus non-normality due to outliers and suboptimal measurement). In Chapter 2, Wiedermann, Li, and von Eye then continue the discussion of asymmetry properties of the linear regression model and introduce three asymmetry concepts (summarized in a framework termed Direction Dependence Analysis (DDA), cf. Wiedermann and von Eye, 2015a; Wiedermann & Li, 2018) that can be used to detect potential confounding and distinguish between the two causally competing models x y and y x. Applications of DDA in the context of mediation and moderation models are discussed. Chapter 3, by Engin A. Sungur, is devoted to the use of copulas in direction dependence modeling. This chapter introduces definitions and fundamental principles to model directional dependence of variables using asymmetric copulas and regression, and describes various copula-based directional dependence measures to perform model selection in both, continuous and categorical data settings.
The second module is devoted to Direction Dependence in Continuous Variables. Chapter 4, by Wolfgang Wiedermann, discusses asymmetry properties of the partial correlation coefficient in the research tradition of Dodge and Rousson (2000). Asymmetric facets of the partial correlation coefficient are presented which enable one to test causally competing models while adjusting for relevant background variables. Parameter recovery and accuracy of model selection is evaluated using Monte-Carlo simulation experiments. Chapter 5, by Shimizu and Blöbaum, gives an overview of recent advances in the development of algorithms for unsupervised causal learning. The authors start by introducing the standard LiNGAM and present extensions to structural vector autoregressive models for the analysis of time series data, models with hidden common causes, and methods for causal learning under nonlinearity of...
