CHAPTER 1

GENERALIZED INVERSE MATRICES

1. INTRODUCTION

The application of generalized inverse matrices to linear statistical models is of relatively recent occurrence. As a mathematical tool such matrices aid in understanding certain aspects of the analysis procedures associated with linear models, especially the analysis of unbalanced data, a topic to which considerable attention is given in this book. An appropriate starting point is therefore a summary of the features of generalized inverse matrices that are important to linear models. Other ancillary results in matrix algebra are also discussed.

a. Definition and existence

A generalized inverse of a matrix A is defined, in this book, as any matrix G that satisfies the equation

(1)

The name “generalized inverse” for matrices G defined by (1) is unfortunately not universally accepted, although it is used quite widely. Names such as “conditional inverse”, “pseudo inverse” and “g-inverse” are also to be found in the literature, sometimes for matrices defined as is G of (1) and sometimes for matrices defined as variants of G. However, throughout this book the name “generalized inverse” of A is used exclusively for any matrix G satisfying (1).

Notice that (1) does not define G as “the” generalized inverse of A but as “a” generalized inverse. This is because G, for a given matrix A, is not unique. As shown below, there is an infinite number of matrices G that satisfy (1) and so we refer to the whole class of them as generalized inverses of A.

One way of illustrating the existence of G and its non-uniqueness starts with the equivalent diagonal form of A. If A has order p × q the reduction to this diagonal form can be written as

or, more simply, as

As usual, P and Q are products of elementary operators [see, for example, Searle (1966), Sec. 5.7], r is the rank of A and Dr is a diagonal matrix of order r. In general, if d1 d2, …, dr, are the diagonal elements of any diagonal matrix D we will use the notation D{di} for Dr; i.e.,

(2)

Furthermore, as in Δ, null matrices will be represented by the symbol 0, with order being determined by context on each occasion.

Derivation of G comes easily from Δ. Analogous to Δ we define Δ− (to be read as “Δ minus”) as

Then, as shown below,

(3)

satisfies (1). Hence G is a generalized inverse of A. Clearly G as given by (3) is not unique, for neither P nor Q by their definition is unique; neither is Δ nor Δ−, and therefore G = QΔ−P is not unique.

Before showing that G does satisfy (1), note from the definitions of Δ and Δ− given above that

(4)

Hence, by the definition implied in (1), we can say that Δ− is a generalized inverse of Δ, an unimportant result in itself but one which leads to G satisfying (1). To show this we use Δ to write

(5)

the inverses P−1 and Q−1 existing because P and Q are products of elementary operators and hence non-singular. Then (3), (4) and (5) give

i.e., (1) is satisfied. Hence G is a generalized inverse of A.

Example. For

a diagonal form is obtained using

so that

Hence

The reader should verify that AGA = A.

It is to be emphasized that generalized inverses exist for rectangular matrices as well as for square ones. This is evident from the formulation of Δpxq. However, for A of order p × q, we define Δ− as having order q × p, the null matrices therein being of appropriate order to make this so. As a result G has order q × p.

Example. Consider

the same as A in the previous example except for an additional column. With P as given earlier and Q now taken as

Δ− is then taken as

b. An algorithm

Another way of computing G is based on knowing the rank of A. Suppose it is r and that A can be partitioned in such a way that its leading r × r minor is non-singular, i.e.,

where A11 is r × r of rank r. Then a generalized inverse of A is

where the null matrices are of appropriate order to make G be q × p. To see that G is a generalized inverse of A, note that

Now, by the way in which A has been partitioned, [A21 A22] = K[A11 A12] for some matrix K. Therefore K = A21A−111 and so A22 = KA12 = A21A−111A12. Hence AGA = A.

Example. A generalized inverse of

There is no need for the non-singular minor of order r to be in the leading position. Suppose it is not. Let R and S represent the elementary row and column operations respectively to bring it to the leading position. Then R and S are products of elementary operators with

where B11 is non-singular of order r. Then

is a generalized inverse of B and Gqxp = SFR is a generalized inverse of A. Now R and S are products of elementary operators that interchange rows (or columns); i.e., R and S are products of matrices that are identity matrices with rows (or columns) interchanged. Therefore R and S are identity matrices with rows (or columns) in a different sequence from that found in I. Such matrices are known as permutation matrices and are orthogonal; i.e.,

and

(6)

The same is true for S, and so from RAS = B we have

Clearly, so far as B11 is concerned, this product represents the operations of returning the elements of B11 to their original positions in A. Now consider G: we have

In this, analogous to the form of A = R′BS′, the product involving R′ and S′ in G′ represents putting the elements of (B−111)’ into the corresponding positions (of G′) that the elements of B11 occupied in A. Hence an algorithm for finding a generalized inverse of A by this method is as follows.

(i) In A, of rank r, find any non-singular minor of order r. Call it M (using the symbol M in place of B11). (ii) Invert M and transpose the inverse: (M−1)′. (iii) In A replace each element of M by the corresponding element of (M−1)′; i.e., if aij = mst, the (s, t)th element of M, then replace aij by mt,s, the (t, s)th element of M−1, equivalent to the (s, t)th element of the transpose of M−1. (iv) Replace all other elements of A by zero. (v) Transpose the resulting matrix. (vi) The result is G, a generalized inverse of A.

Note that this procedure is not equivalent, in (iii), to replacing elements of M in A by the elements of M−1 (and others by zero) and then in (v) transposing. It is if M is symmetric. Nor is it equivalent to replacing, in (iii), elements of M in A by elements of M−1 (and others by zero) and then in (v) not transposing (see Exercise 5). In general, the algorithm must be carried out exactly as described.

One case where it can be simplified is when A is symmetric. Then any principal minor of A is symmetric and the transposing in both (iii) and (v) can be ignored. The algorithm can then become as follows.

(i) In A, of rank r and symmetric, find any non-singular principal minor of order r. Call it M. (ii) Invert M. (iii) In A replace each element of M by the corresponding element of M−1. (iv) Replace all other elements of A by zero. (v) The result is G, a generalized inverse of A.

However, when A is symmetric and a non-symmetric non-principal minor is used for M, then the general algorithm must be used.

Example. The matrix

has the following matrices, among others, as generalized inverses:

derived from inverting the 2 × 2 minors

Similarly,

as a generalized inverse.

These derivations of a matrix G that satisfies (1) are by no means the only ways in which such a matrix can be computed....