Orthogonal projection matrix of a matrix with one column sign switched

Question

Suppose that I have a $n \times k$ real matrix with full column rank. Say $k=3$ and I write

$$X = [\mathbf x_1:\mathbf x_2:\mathbf x_3]$$

where lower-case $\mathbf x$'s are $n \times 1$ vectors.

I go on and form the orthogonal projection matrix

$$P_X = X \left(X'X\right)^{-1}X'$$.

Consider now the matrix

$$W = [\mathbf x_1:-\mathbf x_2:\mathbf x_3]$$

Namely it is equal to $X$ matrix, but in (any) one column, the sign of the elements are switched.

Question: Can we express the projection matrix of $W$, $P_W=W \left(W'W\right)^{-1}W'$, in terms of the projection matrix of $X$, $P_X$, or at least state some relation between them?

I tried to explore this with what little matrix algebra I know, but could not come up with anything. In reality the $k$ dimension is bigger that $3$ but I guess this does not matter.

Answer 1

We have $P_X = P_W$.

We can show this using matrix algebra by noting that $W = XQ$, where $Q$ is the orthogonal matrix $$ Q = \pmatrix{1&0&0\\0&-1&0\\0&0&1}. $$ With that, we note that $$ \begin{align} P_W &= W(W'W)^{-1}W' = [XQ]([XQ]'[XQ])^{-1}[XQ]' \\ & = XQ[Q' (X'X) Q]^{-1}Q'X \\ & = XQ[Q'(X'X)^{-1}Q]Q'X \\ & = X[QQ'](X'X)^{-1}[QQ']X = X(X'X)^{-1}X = P_X, \end{align} $$ as was desired.

Answer 2

The reason why the matrices are the same is because you are projecting on the same subspace. There is no need for a proof here. Even if you replace your columns by some linear combinations that have the same span, you get the same matrix. They do not have to be independent either.

Answer 3

The projection is defined uniquely by the properties that it is the identity map on the column space of $X$ and zero on the orthogonal complement. The matrix that represents a linear transformation in a given basis (here, the standard basis) is unique, so if you use any other full-rank matrix $Y$ that has the same column space as $X$, you’ll end up with the same projection matrix.