Let $Y = X\beta + \varepsilon$, where $\varepsilon$ is multivariate normal with mean of $0$ and variance of $\sigma^2 I$.
Suppose $X = (X_1\ \ X_2)$, and correspondingly, $\beta = (\beta_1\ \ \beta_2)$. Let $\hat{\beta}$ be the OLS estimate of $\beta$. Also, $\|Y\|$ is the Euclidean norm of $Y$, not that it makes a huge difference.
If $X_1,\ X_2$ are orthogonal, then prove the following inequality:
$$\|Y - X_1\hat{\beta}_1\|^2 + \|Y - X_2\hat{\beta}_2\|^2 \geq \|Y - X\hat{\beta}\|^2$$
Here is my progress:
Let $H$ be the hat matrix. Then from triangular inequality:
\begin{align} & \|Y - X_1\hat{\beta}_1\|^2 + \|Y - X_2\hat{\beta}_2\|^2 \\ \geq {} & \|2Y - X_1\hat{\beta}_1-X_2\hat{\beta}_2\|^2 \\ = {} & \|2Y - X\hat{\beta}\|^2 \text{ since } \hat{\beta}^T = (\hat{\beta}^T_1\ \ \hat{\beta}^T_2) \\ = {} & \|2Y-HY\|^2 \\ = {} & \|(2I-H)Y\|^2 \end{align}
I don't know how I can justify $\|(2I-H)\|^2 \geq \|I-H\|^2$.
Am I on the right track? If so, how should I justify the last inequality? If I am not on the right track, please guide me to the right track! Thanks!
$||(2I - H)Y||^2 = ||[I + (I-H)]Y||^2 = ||Y + (I-H)Y||^2 = ||Y + \hat{\epsilon}||^2 \geq ||\hat{\epsilon}||^2 = ||(I-H)Y||^2$
?