Calculated product of two projection matrices

Question

how do i show that $$P_MP_m=P_mP_M=P_m$$ where $$P_M=X_M(X'_MX_M)^{-1}X'_M$$ $$P_M=X_m(X'_mX_m)^{-1}X'_m$$

and $X_m$ is embedded into the larger matrix with the same number of columns $X_M$.

Answer 1

Let $A$ be a tall matrix (i.e., at least as many rows as columns) of full rank (equivalently, $A$ is injective). Then $A^TA$ is invertible. Put $P := A(A^TA)^{-1}A^T$. Then $P$ satisfies $P^2 = P$ and $P^* = P$. Hence, $P$ is an orthogonal projection. The image of $P$ coincides with the image of $A$. This follows from the surjectivity of $A^T$ and $(A^TA)^{-1}$. Thus, $P$ is the orthogonal projection onto the image of $A$.

Now, $X_m$ is $X_M$ with a few columns deleted. Hence, the image of $X_m$ is contained in that of $X_M$. Therefore, the claim reduces to the following: If $M$ and $N$ are subspaces of a Euclidian space (finite-dimensional, here) such that $M\subset N$, then $P_MP_N = P_NP_M = P_M$, where $P_L$ denotes the orthogonal projection onto the subspace $L$. And this is very easily verified. For example, if $x$ is a vector, then $P_Mx\in M$ and thus $P_Mx\in N$. Now, $P_N$ operates as the identity on $N$ (since it projects onto $N$ - so when you are already in $N$ and you apply $P_N$, nothing changes). Hence, $P_NP_Mx = P_Mx$. As this holds for any $x$, we have $P_NP_M = P_M$. For $P_MP_N = P_M$ use that $I - P_N$ is the orthogonal projection onto $N^\perp$.