Some linear algebra fact: Schur complement formula and projection operator

One of the most important thing is Schur complement formula: suppose we have a square {2\times 2}-block matrix of arbitrary block size:

\displaystyle  X = \left(\begin{array}{cc} A & B \\ C & D \end{array} \right) \ \ \ \ \ (30)

then {\det X = \det (A - B D^{-1} C) \det D}. The proof is very simple, and consists of factoring out {D} from {X} and then doing a block column (or row) operation:

  1. \displaystyle  X = \left(\begin{array}{cc} A & BD^{-1} \\ C & I \end{array} \right) \left(\begin{array}{cc} I & 0 \\ 0 & D \end{array} \right) =: X_1 X_2 \ \ \ \ \ (31)

  2. Now we multiply the right column block {(BD^{-1}, I)^t} of {X_1} by {C} on the right (order is important because these are not numbers), and subtract it from the left column block {(A,C)^t}. This gives

    \displaystyle  \det \left(\begin{array}{cc} A & BD^{-1} \\ C & I \end{array} \right) = \det \left(\begin{array}{cc} A- BD^{-1} C & BD^{-1} \\ 0 & I \end{array} \right) \\ = \det (A - BD^{-1} C) \ \ \ \ \ (32)

    The last determinant is readily evaluated since it’s in block upper triangular form.

Next we prove that for any matrix {n \times m} matrix {A} with {m \le n} and {A} full rank, i.e., the columns are independent, the following operator is the orthogonal projection operator onto the column space of {A} (the inner product defining orthogonality is Hermitian, which generalizes the usual dot project in {{\mathbb R}^n}):

\displaystyle  P = A (A^* A)^{-1} A^* \ \ \ \ \ (33)

here {A} can be complex entried, hence we talk about Hermitian conjugate. First note that {P^2 = A(A^* A)^{-1} A^* A (A^* A)^{-1} A^* = P}. Next observe that if {v} is in the orthogonal complement of the column space of {A}, then {A^* v = (\langle a_1, v\rangle, \ldots, \langle a_m, v\rangle)^t = \vec{0}}. So {P v = 0}. This shows {P} is indeed the orthogonal projection onto colspace of {A}, and further,

\displaystyle  Q = I - P \ \ \ \ \ (34)

is the orthogonal projection onto the orthogonal complement of colspace of {A}.
The above result is somewhat miraculous, as one could try to prove it directly when {A} has rank 1 or when its columns are orthogonal. But a direct proof in the general case seems hard.
Finally, how are the previous two results related? Say one has a nonsingular square matrix {X}, and would like to compute the {kk}th entry of its Hermitian square {(X^* X)^{-1}_{kk}}. By the adjoint formula for inverting a matrix we have

\displaystyle  (X^*X)^{-1}_{kk} = \frac{\det (X_{(k)}^* X_{(k)}}{\det (X^* X)} \ \ \ \ \ (35)

where {X_{(k)}} denotes the matrix {X} with the {k}th column removed (not the kth column!). The above (35) is true because the {kk}th adjoint of {X^* X} is exactly {X_{(k)}^* X_{(k)}} and one has to remember to divide by the determinant of the original matrix.
Now we move the {k}th column {X_k} of {X} to the first column and write

\displaystyle  X = \left(\begin{array}{cc} X_k & X_{(k)} \end{array} \right) \ \ \ \ \ (36)

then we see that

\displaystyle  X^* X= \left(\begin{array}{cc} \langle X_k, X_k \rangle & X_k^* X_{(k)} \\ X_{(k)}^* X_k & X_{(k)}^* X_{(k)} \end{array} \right). \ \ \ \ \ (37)

So by Schur complement formula

\displaystyle  \det X^* X = \det( \langle X_k, X_k \rangle - X_k^* X_{(k)} (X_{(k)}^* X_{(k)})^{-1} X_{(k)}^* X_k) \det (X_{(k)}^* X_{(k)} \\ = \det (X_k^* Q_k X_k) \det(X_{(k)}^* X_{(k)})\\ = X_k^* Q_k X_k \det(X_{(k)}^* X_{(k)} )\ \ \ \ \ (38)

where {Q_k = I - X_{(k)} (X_{(k)}^* X_{(k)})^{-1} X_{(k)}^{-1}} is simply the orthogonal projection onto the orthogonal complement of the column space of {X_{(k)}}!

Advertisements

About aquazorcarson

math PhD at Stanford, studying probability
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s