On the projection matrix on sum of two subspaces

Question

Define $Z=(X:Y)$. We know that the projection matrix of $Z$ (say, $P_z$) can be expressed as the sum of the projection matrix of $X$ (say, $P_x$) and that of $(I-P_x)Y$. If the column spaces of $X$ and $Y$ are orthogonal, then $P_z=P_x+P_y$, where $P_y$ is the projection matrix of $Y$.

However, I am interested in the general case when the column spaces of $X$ and $Y$ are not necessarily orthogonal. In this context, is it possible to write $P_z \leq P_x+P_y$? Here, `$A\leq B$' implies that $(B-A)$ is a non-negative definite matrix.

On a related note, we also know that when $P_xP_y=P_yP_x$, then $P_z=P_x+P_y-P_xP_y$. However, I am unable to interpret this condition. Suppose, in a regression context, we have a group of covariates $X=(X_1:X_2)$. What would be the implication of the assumption that the projection matrices corresponding to $X_1$ and $X_2$ commute?

Edit: By projection matrix of $X$, I mean the orthogonal projection matrix on the column space of $X$. Here $X,Y$ are non-random. Columns of $X,Y$ are sub-spaces of $\mathbb{R}^n$.

Answer 1

We clearly have that $P_z-P_x$ and $P_z-P_y$ are projections and $P_z$ commutes with $P_x$ and $P_y.$

We are going to prove that $$P_z\le P_x+P_y \iff P_xP_y=P_yP_x\quad (*)$$ We start with proving the fact that for two orthogonal projections $ P$ and $Q$ we have $$ P\le Q\implies PQ=QP\qquad (**)$$ Indeed, we have $0\le I-Q\le I-P.$ Thus $v=Pv$ implies $v=Qv.$ Hence $QP=PQP$ and $PQ=(QP)^*=PQP.$ Thus $PQ=QP.$

Assume $P_z\le P_x+P_y.$ Then $P_z-P_y\le P_x.$ By $(**)$ we get that $P_x$ and $P_z-P_y$ commute. Hence $P_x$ and $P_y$ commute.

The $\Leftarrow$ direction of $(*)$ follows from the identity $$P_z=P_x+P_y-P_xP_y$$ as the operator $P_xP_y$ is positive.

A concrete example can be made up in the two dimensional space by taking two non orthogonal and not parallel vectors $x$ and $y$ and projections on them. Then $P_x$ and $P_y$ do not commute therefore $P_x+P_y\ge P_z$ fails.

Answer 2

The inequality is true if and only if $$ \operatorname{im}(Z)\cap\operatorname{im}(X)^\perp\subseteq\operatorname{im}(Y).\tag{1} $$ (Remark. The asymmetry of the roles of $X$ and $Y$ above is only apparent. Since both $\operatorname{im}(X)$ and $\operatorname{im}(Y)$ are subspaces of $\operatorname{im}(Z)$, the condition is equivalent to $\operatorname{im}(Z)\cap\operatorname{im}(Y)^\perp\subseteq\operatorname{im}(X)$.)

Suppose $(1)$ does not hold. Then there exists a nonzero vector $z_1\in\operatorname{im}(Z)\cap\operatorname{im}(X)^\perp\setminus\operatorname{im}(Y)$. Hence $P_xz_1=0,\,P_yz_1\ne z_1=P_zz_1$, $$ \langle z_1,(P_x+P_y-P_z)z_1\rangle=0+\|P_yz_1\|^2-\|z_1\|^2=-\|(I-P_y)z_1\|^2<0 $$ and $P_x+P_y-P_z$ is not positive semidefinite.

Now suppose $(1)$ holds. For any $v\in\mathbb R^n$, let $$ \begin{aligned} x_1&:=P_xv\in\operatorname{im}(X),\\ y_1&:=(P_z-P_x)v=P_z(I-P_x)v\in\operatorname{im}(Z)\cap\operatorname{im}(X)^\perp\subseteq\operatorname{im}(X)^\perp\cap\operatorname{im}(Y),\\ w_1&:=(I-P_z)v\in\operatorname{im}(Z)^\perp.\\ \end{aligned} $$ Then $v=x_1+y_1+w_1$ and $$ \begin{aligned} \langle v,P_xv\rangle &=\langle x_1+y_1+w_1,\, x_1\rangle=\langle x_1,\,x_1\rangle,\\ \\ \langle v,P_yv\rangle &=\langle x_1+y_1+w_1,\, P_yx_1+y_1\rangle\\ &=\langle x_1,P_yx_1\rangle +\langle x_1,y_1\rangle +\langle y_1,P_yx_1\rangle +\langle y_1,y_1\rangle\\ &=\langle x_1,P_y^2x_1\rangle +0 +\langle P_yy_1,x_1\rangle +\langle y_1,y_1\rangle\\ &=\langle P_yx_1,P_yx_1\rangle +0 +\langle y_1,x_1\rangle +\langle y_1,y_1\rangle\\ &=\langle P_yx_1,P_yx_1\rangle +0 +0 +\langle y_1,y_1\rangle\\ &=\langle P_yx_1,P_yx_1\rangle +\langle y_1,y_1\rangle,\\ \\ \langle v,P_zv\rangle &=\langle x_1+y_1+w_1,\, x_1+y_1\rangle =\langle x_1,\,x_1\rangle+\langle y_1,\,y_1\rangle.\\ \end{aligned} $$ Hence $\langle v,(P_x+P_y-P_z)v\rangle=\langle P_yx_1,P_yx_1\rangle\ge0$. Since $v$ is arbitrary, $P_x+P_y\ge P_z$.