What is that matrix?

Question

Let an inner product on $\mathbb{R}^n$ be given by its Gramian matrix $G$. Let $A:\mathbb{R}^n \rightarrow \mathbb{R}^k$ be a linear operator with $\mathop{\rm rank} A=k$ (We denote its matrix also by $A$.). What is the matrix of the orthogonal projector from $ \mathbb{R}^n$ on $ \ker A$?

Answer 1

We minimize $(x-a)^t G (x-a)$ for some fixed $a$ and $x\in\ker A$. Let $N$ be a matrix whose columns form a basis of $\ker A$. Then, we can equivalently minimize $(Nu-a)^t G (Nu-a)$ for every $u$. The unique minimizer is given by $N^t G (Nu-a) = 0$ or $u = (N^t G N)^{-1} N^t G a$.

The projection matrix is then $N (N^t G N)^{-1} N^t G$, which is obviously not zero.

--

Notice that the range of $G^{-1}A^t$ is orthogonal to kernel of $A$ and has a basis given by the columns of $G^{-1}A^t$. So the projection of the range would be $G^{-1}A^t (AG^{-1}A^t)^{-1} A$ and of its orthogonal complement would be $I - G^{-1}A^t (AG^{-1}A^t)^{-1} A$.

--

For $x\in\ker A$ we have $x^t G G^{-1} A^t = 0$. That is, the kernel of $A$ is orthogonal to range $G^{-1}A^t$. By a dimension argument, the kernel is in fact the orthogonal complement.

Answer 2

An alternate approach: once again, we take $N$ to be a matrix whose columns form a basis of $\ker A$. Let $G^{1/2}$ denote the positive definite square root of $G$.

Note that the columns of $G^{-1/2}$ form an orthonormal basis with respect to the inner product. Call this basis $\mathcal B$. The columns of the matrix $G^{1/2}N$ are the column-vectors of $N$, rewritten relative to $\mathcal B$.

By the formula given here, we can write the matrix of the projection relative to $\mathcal B$ as $$ [P_N]_{\mathcal B} = [G^{1/2}N]([G^{1/2}N]^T[G^{1/2}N])^{-1}[G^{1/2}N]^T = G^{1/2}N(N^TN)^{-1}N^TG^{1/2} $$ The projection relative to the original basis is then given by $$ P_N = G^{-1/2}[P_N]_{\mathcal B}G^{1/2} = N(N^TN)^{-1}N^TG $$