Suppose $X$ and $Z$ are matrices such that $(X,Z)$ and $P_XZ$ both have full column ranks. Here, $P_X=X(X'X)^{-1}X'$. Consider a regression model $$ P_Xy=P_XZ\zeta+v\tag{A} $$ where OLS is used to estimate $\zeta$. After playing around with numbers, it seems that (A) always yields $0$ residuals. Why is this the case?
Attempt: The OLS estimator for $\zeta$ from (A) is $$ (Z'P_XZ)^{-1}Z'P_Xy $$ and so the OLS residuals from (A) is $$ P_Xy-P_XZ(Z'P_XZ)^{-1}Z'P_Xy=[P_X-P_XZ(Z'P_XZ)^{-1}Z'P_X]y \tag{B} $$ For (B) to be identically $0$ for all $y$, it must be that the expression inside the square brackets is zero. But this is where I am stuck.
This is not true in general. An easy counterexample is when $Z$ orthogonal to $X$, and the residue is just $P_Xy$ which does not vanish. This is true only when the $P_Xy$ happens to be in the projected space or the space spanned by $P_XZ$.
The essential point is that OLS is a projection onto a subspace. Thus once the subspace to be projected onto -- no matter what basis is used to represent the subspace and the projection --- is determined, the resulting vector and therefore residue is determined.
With this in mind, we can derived the pertinent result or counterexamples, depending on the given conditions, in two ways, one algebraic and one geometric.
If $X$ is of full column rank $n$, there exist, $U$ the columns of which are orthonormal, and a non-singular square matrix $A$ with the same number of columns as $X$, such that $X=UA$. So $P_X=UU'$. If $X$ and $Z$ have the same number of columns and $P_XZ=UU'Z$ is of full column rank, then $U$ and $Z$ have the same number of columns and rank and $U'Z$ is a full rank (thus invertible) square matrix. Therefore $$(Z'P_XZ)^{-1}=((U'Z)'U'Z)^{-1}=(U'Z)^{-1}\big((U'Z)'\big)^{-1}$$ and $$P_XZ(Z'P_XZ)^{-1}Z'P_X = U(U'Z)(U'Z)^{-1}\big((U'Z)'\big)^{-1}(U'Z)'U'=UU'=P_X$$ which is the desired result.
The result in the above paragraph is not surprising and can be deduced more directly and geometrically as follows. Let $S$ be the space spanned by the columns of $X$. Under the condition in the second paragraph, the columns of $X$ are independent. $P_X$ is the projection matrix onto the space spanned by the columns of $X$. $P_XZ$ consists columns entirely in $S$. $Z$ has the same number of columns as $X$ and $P_XZ$ is of full column rank implies that the columns of $P_XZ$ are also independent and constitutes just another basis of $S$. In other words, the space spanned by the columns of $P_XZ$ is not a proper subspace but the whole $S$. So the projection by $P_XZ$ is the same as projection by $P_X$. $P_Xy$ is already in $S$, its projection by $P_XZ$ onto the original space $S$ is surely itself $P_Xy$. So there is no residue.