CHAPTER ONE

Introduction

Most researchers applying statistics think in terms of modeling the individual observations. In multiple regression or ANOVA (analysis of variance), for instance, we learn that the regression coefficients or the error variance estimates derive from the minimization of the sum of squared differences of the predicted and observed dependent variable for each case. Residual analyses display discrepancies between fitted and observed values for every member of the sample.

The methods of this book demand a reorientation. The procedures emphasize covariances rather than cases.1 Instead of minimizing functions of observed and predicted individual values, we minimize the difference between the sample covariances and the covariances predicted by the model. The observed covariances minus the predicted covariances form the residuals. The fundamental hypothesis for these structural equation procedures is that the covariance matrix of the observed variables is a function of a set of parameters. If the model were correct and if we knew the parameters, the population covariance matrix would be exactly reproduced. Much of this book is about the equation that formalizes this fundamental hypothesis:

(1.1)

In (1.1), ∑ (sigma) is the population covariance matrix of observed variables, θ (theta) is a vector that contains the model parameters, and ∑ (θ) is the covariance matrix written as a function of θ. The simplicity of this equation is only surpassed by its generality. It provides a unified way of including many of the most widely used statistical techniques in the social sciences. Regression analysis, simultaneous equation systems, confirmatory factor analysis, canonical correlations, panel data analysis, ANOVA, analy-sis of covariance, and multiple indicator models are special cases of (1.1).

Let me illustrate. In a simple regression equation we have y = γx + ζ, where γ (gamma) is the regression coefficient, ζ (zeta) is the disturbance variable uncorrelated with x and the expected value of ζ, E(ζ), is zero.The y, x, and ζ are random variables. This model in terms of (1.1) is2

(1.2)

where VAR( ) and COV( ) refer to the population variances and covari- ances of the elements in parentheses. In (1.2) the left-hand side is 2, and the right-hand side is 2(6), with 0 containing γ, VAR(x), and VAR(ζ) as parameters. The equation implies that each element on the left-hand side equals its corresponding element on the right-hand side. For example, COV(x, y) = γVAR(x) and VAR(γ) = y2 VAR(x) + VAR(ζ). I could modify this example to create a multiple regression by adding explanatory variables, or I could add equations and other variables to make it a simultaneous equations system such as that developed in classical econo- metrics. Both cases can be represented as special cases of equation (1.1), as I show in Chapter 4.

Instead of a regression model, consider two random variables, xl and x2, that are indicators of a factor (or latent random variable) called ξ (xi). The dependence of the variables on the factor is x1 = ζ + δ1 and x2 = ξ + δ2, where (delta) and δ2 are random disturbance terms, uncorrelated with ξ and with each other, and E(δ1 ) = E(δ2) = 0. Equation (1.1) specializes to

(1.3)

where ϕ (phi) is the variance of the latent factor ξ. Here θ consists of three elements: ϕ, VAR(δX), and VAR(δ2). The covariance matrix of the observed variables is a function of these three parameters. I could add more indicators and more latent factors, allow for coefficients ("factor loadings") relating the observed variables to the factors, and allow correlated disturbances creating an extremely general factor analysis model. As Chapter 7 demonstrates, this is a special case of the covariance structure equation (1.1).

Finally, a simple hybrid of the two preceding cases creates a simple system of equations. The first part is a regression equation of y = γξ + ζ, where unlike the previous regression the independent random variable is unobserved. The last two equations are identical to the factor analysis example: x1 = ξ+ δ1 and x2 = ξ+ δ2. I assume that ζ, and δ2 are uncorrected with ξ and with each other, and that each has an expected value of zero. The resulting structural equation system is a combination of factor analysis and regression-type models, but it is still a specialization of (1.1):

(1.4)

These examples foreshadow the general nature of the models I treat. My emphasis is on systems of linear equations. By linear, i mean that the relations between all variables, latent and observed, can be represented in linear structural equations or they can be transformed to linear forms.3 Structural equations that are nonlinear in the parameters are excluded. Nonlinear functions of parameters are, however, common in the covariance structure equation, ∑ = ∑(θ). For instance, the last example had three linear structural equations: y = γξ + ξ, x1 = ξ + δ1, and x2 = ξ + δ2. Each is linear in the variables and parameters. Yet the covariance structure (1.4) for this model shows that COV(x1; y) = γϕ, which means that the COV(x1,y) is a nonlinear function of γ and ϕ. Thus it is the structural equations linking the observed, latent, and disturbance variables that are linear, and not necessarily the covariance structure equations.

The term “structurai” stands for the assumption that the parameters are not just descriptive measures of association but rather that they reveal an invariant “causal” relation. I will have more to say about the meaning of “causality” with respect to these models in Chapter 3, but for now, let it suffice to say that the techniques do not “discover” causal relations. At best they show whether the causal assumptions embedded in a model match a sample of data. Also the models are for continuous latent and observed variables. The assumption of continuous observed variables is violated frequently in practice. In Chapter 9 I discuss the robustness of the standard procedures and the development of new ones for noncontinuous variables.

Structural equation models draw upon the rich traditions of several disciplines. I provide a brief description of their origins in the next section.

HISTORICAL BACKGROUND

Who invented general structural equation models? There is no simple answer to this question because many scholars have contributed to their development. The answer to this question is further complicated in that the models continue to unfold, becoming more general and more flexible. However, it is possible to outline various lines of research that have contributed to the evolution of these models.

My review is selective. More comprehensive discussions are available from the perspectives of sociology (Bielby and Hauser 1977), psychology (Bentler 1980; 1986), and economics (Goldberger 1972; Aigner et al. 1984). Two edited collections that represent the multidisciplinary origins of these techniques are the volumes by Goldberger and Duncan (1973) and Blalock ([1971] 1985). Other more recent collections are in Aigner and Goldberg (1977), Jöreskog and Wold (1982), the November 1982 issue of the Journal of Marketing Research, and the May–June 1983 issue of the Journal of Econometrics.

I begin by identifying three components present in today’s general structural equation models: (1) path analysis, (2) the conceptual synthesis of latent variable and measurement models, and (3) general estimation procedures. By tracing the rise of each component, we gain a better idea about the origins of these procedures.

Let me consider path analysis first. The biometrician Sewall Wright (1918, 1921, 1934, 1960) is its inventor. Three aspects of path analysis are the path diagram, the equations relating correlations or covariances to parameters, and the decomposition of effects. The first aspect, the path diagram, is a pictorial representation of a system of simultaneous equations. It shows the relation between all variables, including disturbances and errors. Figure 1.1 gives a path diagram for the last example of the previous section. It corresponds to the equations:

where ξ, δ1, and δ2 are uncorrelated with each other and with ξ. Straight single-headed arrows represent one-way causal influences from the variable at the arrow base to the variable to which the arrow points. The implicit coefficients of one for the effects of ξ on x1 and x2 are made explicit in the diagram.

Figure 1.1 Path Diagram Example

Using the path diagram, Wright proposed a set of rules for writing the equations that relate the correlations (or covariances) of variables to the model parameters; this constitutes the second aspect of path analysis. The equations are equivalent to covariance structure equations, an example of...