Prove that $J=JH=HJ$

Question

The question asks for the idempotency of \begin{align*} H-\frac{1}{n}J \end{align*} where $H$ is the hat matrix for multilinear regression, $H = X(X'X)^{-1}X'$, and $J$ is the matrix of all 1's.

I proceeded by definition, i.e., wanting to show that $(H-\frac{1}{n}J)(H-\frac{1}{n}J)= H-\frac{1}{n}J$, expanding it I got: \begin{align*} H-\frac{1}{n}(HJ+JH)+\frac{1}{n}J \end{align*} where we can find that $HJ=JH$, knowing that any element of the i-th row of $HJ$ is the sum of i-th row of $H$; any element of j-th column of $JH$ is the sum of j-th column of $H$, combining with the fact that $H$ is symmetric.

How can I proceed from up here? It seems necessary that $HJ=J$, but how do we prove it?

Answer 1

In order to have $JH = HJ = J$ we need to have that $\mathbf{1}$ be an eigenvector of $H$ since $R(J) = \operatorname{span}(\mathbf{1})$, where $\mathbf{1}$ is a vector of all ones and $H$ is a projector (has eigenvalues $1$ or $0$). Since $H$ is an orthogonal projector onto $R(X)$, i.e. $R(X) = R(H)$, it follows that your data must contain a column of ones. In that case, since $H$ is a projector, it follows that $HX = X \implies H\mathbf{1}=\mathbf{1}$, thus $HJ = JH = J$.

Answer 2

Supposing that $X$ is a design matrix as in linear regression with an intercept, then $X$ has a column of ones. In particular $$ HJ=\pmatrix{ H\mathbf{1}\mid H \mathbf{1}\mid\dotsb\mid H \mathbf{1} }=J $$ since $\mathbf{1}$ is in the column space of $X$ and $H$ is the projection matrix onto the column space of $X$, where the dividers indicate the columns of $HJ$.