Does a projection matrix mean $P=P^T$?

Question

I know that a matrix $P$ is a projection matrix IFF $P=P^2$, but is it also true that it must be such that $P=P^T$?

I thought that was only true for orthogonal projection matrices.

But i am reading https://see.stanford.edu/materials/lsoeldsee263/homeworkProblems.pdf on page 24, exercise 4.3 it says that BOTH must be true for it to be a projection matrix. Is that correct?

Answer 1

It is true that the literature is not completely clear regarding the distinction between general projections and orthogonal projections.

A projection in $\mathbb{R}^n$ is a linear transformation $P:\mathbb{R}^n\to\mathbb{R}^n$ such if $M$ is the range of $P$, then $Px=x$ for every $x\in M$.

It follows immediately from the definition that if $P$ is a projection, then $P^2=P$, i.e., $P$ is an idempotent. Conversely, if $P$ is and idempotent, then with $M=\hbox{Im}P$, one has clearly $Px=x$ for every $x\in M$, so that the terms "idempotent" and "projection" are equivalent.

An orthogonal projection is a projection $P$ in $\mathbb{R}^n$ that has the additional property: for every $x\in\mathbb{R}^n$, the vector $x-Px$ is orthogonal to $M$, the range of $P$.

The example given by MightTyGuy shows that a general projection need not be symmetric; an orthogonal projection, however, must be symmetric:

Proposition Every orthogonal projection is symmetric.

Proof Let $M$ denote the range (image) of $P$, and let $M^{\perp}$ denote the orthogonal complement of $M$ in $\mathbb{R}^n$. Let $m=\hbox{dim}M$. If $m=n$, then $M=\mathbb{R}^n$, so from $Px=x$ for every $x\in M$ it follows that $P$ is the identity transformation, hence symmetric. Assume $1\leq m <n$. Choose an orthonormal basis $\{u_i\}_{i=1}^n$ in $\mathbb{R}^n$ such that $\{u_1,\dots,u_m\}$ is an orthonormal basis for $M$ and $\{u_{m+1},\dots,u_n\}$ is an orthonormal basis for $M^{\perp}$.

Note that for every $x\in M^{\perp}$, we have $Px=0$. In fact, since $x-Px$ is orthogonal to $Px$, we have, for every $x\in\mathbb{R}^n$: $$0=\langle x-Px,Px\rangle = \langle x,Px\rangle -\langle Px,Px\rangle = \langle x,Px\rangle-\|Px\|^2$$ In particular, if $x\in M^{\perp}$ then $\langle x,Px\rangle=0$, hence $Px=0$. It follows that for every $x\in\mathbb{R}^n$: $$Px=P\left(\sum_{I=1}^n\langle x,u_i\rangle u_i\right)=\sum_{I=1}^m\langle x,u_i\rangle u_i$$ Hence, for every $x,y\in\mathbb{R}^n$: $$\langle Px,y\rangle=\langle\sum_{i=1}^m\langle x,u_i\rangle u_i,\sum_{i=1}^n\langle y,u_i\rangle u_i\rangle=\sum_{i=1}^m\langle x,u_i\rangle\langle y,u_i\rangle$$ and on the other hand $$\langle x,Py\rangle=\langle\sum_{i=1}^n\langle x,u_i\rangle u_i,\sum_{i=1}^m\langle y,u_i\rangle u_i\rangle=\sum_{i=1}^m\langle x,u_i\rangle\langle y,u_i\rangle$$ So that $\langle Px,y\rangle=\langle x,Py\rangle$ for every $x,y\in\mathbb{R}^n$, hence $P$ is symmetric.

Answer 2

It is a definition, so of course one could choose to define a projection with both properties or with just one. I've seen authors use both. Requiring both properties to hold tends to be a more common approach. Consider the matrix $$P=\begin{bmatrix}1&0\\1&0\end{bmatrix}$$ Now $P^2=P$, but $P\ne P^T$. If we consider the action of $P$ on some vector, say $P(2,3)=(2,2)$, then we see that the "error" vector $(0,1)$ is not orthogonal to the projected vector, $(2,2)$. This is a property that is generally desirable for geometric projections, so a common convention requires $P=P^T$ in order for a matrix to qualify as a projection.

But really, it is a matter of convention. Do you, as an author, want your projections to be orthogonal, or not?

Answer 3

If $V$ is a vecor space and if $P:V \to V$ is linear, then $P$ is called a projection if $P^2=P$.

If, in addition, $V$ is a reall or complex vector space with inner product and $P$ is as above, then $P$ is called an orthogonal projection if $ im(P) \perp ker(P)$.

Then is is easy to see that a projection is an orthogonal projection $ \iff P=P^T$.