An algebraic identity: $M(X_2)-M(X)=P(M(X_2)X_1)$?

51 Views Asked by At

Notation: for a matrix $Z$ of full column rank, we define $P(Z)=Z(Z'Z)^{-1}Z'$ and $M(Z)=I-P(Z)$. ($I$ is the identity of matrix with as many rows as $Z$.)

Let's consider the linear regression model with fixed regressors: $$ y=\underbrace{X_1}_{T\times k_1}\beta_1+\underbrace{X_2}_{T\times k_2}\beta_2+u,\quad u\sim N(0,\sigma^2 I_T). $$ With $X=(X_1\;\; X_2)$ (dimension $n\times(k_1+k_2)$), the $F$ statistic for the null hypothesis $\beta_1=\gamma$ can be written in the following equivalent forms: (below $\hat{\beta}_1$ denotes the OLS estimator of $\beta_1$) \begin{align*} F&\equiv\frac{(\hat\beta_1-\gamma)'X_2'M(X_1)X_2(\hat\beta_1-\gamma)/k_1}{y'M(X)y/(T-k_1-k_2)}\\ &=\frac{u'(M(X_2)-M(X))u/k_1}{u'M(X)u/(T-k_1-k_2)}\\ &=\frac{u'P(M(X_2)X_1)u/k_1}{u'M(X)u/(T-k_1-k_2)}\cdot \end{align*} Given the equality of the expressions on the second and third lines above, I suspect $$ M(X_2)-M(X)\overset{?}{=}P(M(X_2)X_1)=M(X_2)X_1(X_1'M(X_2)X_1)^{-1}X_1'M(X_2).\tag{$*$} $$ I tried proving it with various algebraic manipulations but have been hitting dead ends so far. Thanks for your help.


Edit: Writing $M(X_2)=I-P(X_2)$, we can transform ($*$) as $$ M(X_2)-M(X)=P(M(X_2)X_1)\iff P(X)=P(X_2)+P(M(X_2)X_1)\tag{$**$}. $$ ($**$) has a geometric interpretation: projecting on the column space of $X_1$ and $X_2$ can be done in 2 steps: projecting first on $X_2$ and add that to the projection on the "marginal" contribution of $X_1$.

1

There are 1 best solutions below

0
On BEST ANSWER

I think I've found a proof. Using the block inversion formula, we can write $$ P(X)=X_1(X_1'M(X_2)X_1)^{-1}X_1M(X_2)+X_2(X_2'M(X_1)X_2)^{-1}X_2M(X_1). $$ Because $I=P(X_2)+M(X_2)$ and $X_2=P(X_2)X_2$, we can write \begin{align*} P(X)&=M(X_2)X_1(X_1'M(X_2)X_1)^{-1}X_1M(X_2)+P(X_2)X_1(X_1'M(X_2)X_1)^{-1}X_1M(X_2)\\ &\quad+P(X_2)X_2(X_2'M(X_1)X_2)^{-1}X_2M(X_1)\\ &=P(M(X_2)X_1)\\ &\quad+P(X_2)[X_1(X_1'M(X_2)X_1)^{-1}X_1M(X_2)+X_2(X_2'M(X_1)X_2)^{-1}X_2M(X_1)]\\ &=P(M(X_2)X_1)+P(X_2)P(X)\\ &=P(M(X_2)X_1)+P(X_2) \end{align*} where the last equality is because $$ P(X)P(X_2)=P(X)X_2(X_2'X_2)^{-1}X_2=X_2(X_2'X_2)^{-1}X_2=P(X_2). $$ I'll leave this answer here because it seems correct but it's very grungy because it involves the block inversion formula. I hope to learn a more elegant proof from the community.