## Some linear algebra fact: Schur complement formula and projection operator

One of the most important thing is Schur complement formula: suppose we have a square ${2\times 2}$-block matrix of arbitrary block size:

$\displaystyle X = \left(\begin{array}{cc} A & B \\ C & D \end{array} \right) \ \ \ \ \ (30)$

then ${\det X = \det (A - B D^{-1} C) \det D}$. The proof is very simple, and consists of factoring out ${D}$ from ${X}$ and then doing a block column (or row) operation:

1. $\displaystyle X = \left(\begin{array}{cc} A & BD^{-1} \\ C & I \end{array} \right) \left(\begin{array}{cc} I & 0 \\ 0 & D \end{array} \right) =: X_1 X_2 \ \ \ \ \ (31)$

2. Now we multiply the right column block ${(BD^{-1}, I)^t}$ of ${X_1}$ by ${C}$ on the right (order is important because these are not numbers), and subtract it from the left column block ${(A,C)^t}$. This gives

$\displaystyle \det \left(\begin{array}{cc} A & BD^{-1} \\ C & I \end{array} \right) = \det \left(\begin{array}{cc} A- BD^{-1} C & BD^{-1} \\ 0 & I \end{array} \right) \\ = \det (A - BD^{-1} C) \ \ \ \ \ (32)$

The last determinant is readily evaluated since it’s in block upper triangular form.

Next we prove that for any matrix ${n \times m}$ matrix ${A}$ with ${m \le n}$ and ${A}$ full rank, i.e., the columns are independent, the following operator is the orthogonal projection operator onto the column space of ${A}$ (the inner product defining orthogonality is Hermitian, which generalizes the usual dot project in ${{\mathbb R}^n}$):

$\displaystyle P = A (A^* A)^{-1} A^* \ \ \ \ \ (33)$

here ${A}$ can be complex entried, hence we talk about Hermitian conjugate. First note that ${P^2 = A(A^* A)^{-1} A^* A (A^* A)^{-1} A^* = P}$. Next observe that if ${v}$ is in the orthogonal complement of the column space of ${A}$, then ${A^* v = (\langle a_1, v\rangle, \ldots, \langle a_m, v\rangle)^t = \vec{0}}$. So ${P v = 0}$. This shows ${P}$ is indeed the orthogonal projection onto colspace of ${A}$, and further,

$\displaystyle Q = I - P \ \ \ \ \ (34)$

is the orthogonal projection onto the orthogonal complement of colspace of ${A}$.
The above result is somewhat miraculous, as one could try to prove it directly when ${A}$ has rank 1 or when its columns are orthogonal. But a direct proof in the general case seems hard.
Finally, how are the previous two results related? Say one has a nonsingular square matrix ${X}$, and would like to compute the ${kk}$th entry of its Hermitian square ${(X^* X)^{-1}_{kk}}$. By the adjoint formula for inverting a matrix we have

$\displaystyle (X^*X)^{-1}_{kk} = \frac{\det (X_{(k)}^* X_{(k)}}{\det (X^* X)} \ \ \ \ \ (35)$

where ${X_{(k)}}$ denotes the matrix ${X}$ with the ${k}$th column removed (not the kth column!). The above (35) is true because the ${kk}$th adjoint of ${X^* X}$ is exactly ${X_{(k)}^* X_{(k)}}$ and one has to remember to divide by the determinant of the original matrix.
Now we move the ${k}$th column ${X_k}$ of ${X}$ to the first column and write

$\displaystyle X = \left(\begin{array}{cc} X_k & X_{(k)} \end{array} \right) \ \ \ \ \ (36)$

then we see that

$\displaystyle X^* X= \left(\begin{array}{cc} \langle X_k, X_k \rangle & X_k^* X_{(k)} \\ X_{(k)}^* X_k & X_{(k)}^* X_{(k)} \end{array} \right). \ \ \ \ \ (37)$

So by Schur complement formula

$\displaystyle \det X^* X = \det( \langle X_k, X_k \rangle - X_k^* X_{(k)} (X_{(k)}^* X_{(k)})^{-1} X_{(k)}^* X_k) \det (X_{(k)}^* X_{(k)} \\ = \det (X_k^* Q_k X_k) \det(X_{(k)}^* X_{(k)})\\ = X_k^* Q_k X_k \det(X_{(k)}^* X_{(k)} )\ \ \ \ \ (38)$

where ${Q_k = I - X_{(k)} (X_{(k)}^* X_{(k)})^{-1} X_{(k)}^{-1}}$ is simply the orthogonal projection onto the orthogonal complement of the column space of ${X_{(k)}}$!