Jump to ContentJump to Main Navigation
Procrustes Problems$

John C Gower and Garmt B Dijksterhuis

Print publication date: 2004

Print ISBN-13: 9780198510581

Published to Oxford Scholarship Online: September 2007

DOI: 10.1093/acprof:oso/9780198510581.001.0001

Show Summary Details
Page of

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2016. All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a monograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: null; date: 06 December 2016

(p.203) Appendix C Orthogonal projections

(p.203) Appendix C Orthogonal projections

Source:
Procrustes Problems
Publisher:
Oxford University Press

The orthogonal projection B of a point A onto a linear space L is such that the direction AB is orthogonal to all directions in the space; it follows that B is the nearest point in L to A. Figure C.1 illustrates the geometry:

If A 1 has coordinates x and B 1 has coordinates y and the columns of P L R are independent and span the R-dimensional subspace L, then because A 1 B 1 is orthogonal to L, (xy)L = 0 and because yL, there exists an R-dimensional column-vector c such that y′ = Lc. It follows that:

c = ( L L ) 1 L x
and
(C.1)
y = x L ( L L ) 1 L .
Equation (C.1) is the general algebraic form for representing orthogonal projections. However, if we choose the columns of L to be orthonormal, so that L′L = I, we have the simplified form:
(C.2)
y = x L L .
Equations (C.1) and (C.2) give the coordinates of the point B relative to the original P-dimensional axes. Relative to the R orthonormal axes in L, (C.2) becomes;
(C.3)
y = x L .
Appendix C Orthogonal projections

Fig. C.1. Illustration of the orthogonal projection of a point A 1 not lying in L onto a point B 1 in the subspace L.

(p.204) Both the forms (C.2) and (C.3) will be required, and the reader is warned that there is often some confusion because it is not made sufficiently clear which is the reference coordinate system, the original one in P dimensions or a coordinate system embedded in the R-dimensional subspace.

When the columns of L are orthonormal we may regard them as the first R columns of some P-dimensional orthogonal matrix whose final PR columns are irrelevant, so long as they are normal to each other and to the first R columns. Thus, if we partition an orthogonal matrix Q = (Q 1, Q 2), then we have:

x Q = ( x Q 1 , x Q 2 )
showing the rotation, or reflection, as the concatenation of two projections onto orthogonal subspaces.

We note that we must have RP p. We have seen that R = P gives an orthogonal matrix. It is impossible to have L′L = I when R > P because, geometrically, it would require the existence of R orthogonal directions in a space of fewer than R dimensions.

Returning to the general form (C.1) we shall write:

(C.4)
P = L ( L L ) 1 L
for the projection matrix. We note that P 2 = P, so that the matrix is idempotent. This means that once A is projected to B, repeated projection will leave the point B unchanged, which is obvious geometrically.

C.1 Orthonormal operators as rotations

Above, xL was interpreted as a projection from a P-dimensional space onto a subspace spanned by the Q orthogonal columns of L. What might be the interpretation of xL′? Because Q < P, this certainly means that a point in q dimensions is transformed into a point in a higher dimensional space of P dimensions. The columns of L′ are not orthonormal, so xL′ does not represent projection. We have that (xL′)(xL′)′ = xx′, so inner products are invariant to the transformation and, hence, so are distances. Unlike with projections, xL′ preserves configurations and hence it must represent a generalised rotation of x into the higher dimensional space. Another way of seeing this is to write:

x L = ( x 0 ) ( L L )
where x is padded out by PQ extra zero coordinates and L┴ represents a set of orthogonal columns in the orthogonal complement of L. Now (L, L ) is an orthogonal matrix of order P and so represents an orthogonal rotation in P dimensions.