Why is a projection matrix symmetric?

Question

I am looking for an intuitive reason for a projection matrix of an orthogonal projection to be symmetric. The algebraic proof is straightforward yet somewhat unsatisfactory.

Take for example another property: $P=P^2$. It's clear that applying the projection one more time shouldn't change anything and hence the equality.

So what's the reason behind $P^T=P$?

Answer 1

In general, if $P = P^2$, then $P$ is the projection onto $\operatorname{im}(P)$ along $\operatorname{ker}(P)$, so that $$\mathbb{R}^n = \operatorname{im}(P) \oplus \operatorname{ker}(P),$$ but $\operatorname{im}(P)$ and $\operatorname{ker}(P)$ need not be orthogonal subspaces. Given that $P = P^2$, you can check that $\operatorname{im}(P) \perp \operatorname{ker}(P)$ if and only if $P = P^T$, justifying the terminology "orthogonal projection."

Answer 2

In the simple case where $P$ is projection onto a line with unit vector $\vec{e}$, you can look at it like this: if $\theta$ is the angle between $x$ and $\vec{e}$, and $\varphi$ is the angle between $y$ and $\vec e$, we have:

$$\langle Px,y \rangle = \langle \|x\| \cos{\theta \vec{e}},y \rangle = \|x\|\cos{\theta}\langle \vec{e},y \rangle = \|x\|\cos{\theta}\|y\|\cos{\varphi} \\ \langle x, Py \rangle = \langle x,\|y\|\cos{\varphi}\vec{e}\rangle = \|y\| \cos {\varphi} \langle x,\vec{e} \rangle = \|y\|\cos{\varphi}\|x\| \cos{\theta}.$$

(Recall that $P$ is symmetric iff $\langle Px, y \rangle = \langle x, Py \rangle$ for all $x,y \in V$.)

Another reason: Given a projection (orthogonal or not), we can describe it as the projection onto $A$ along $B$ (or $\rho_{A,B}$), where $A$ and $B$ are complementary subspaces. That is, if $V = A \oplus B$, take a basis for the vector space passing through the subspaces $A$ and $B$. Then the action of $P = \rho_{A,B}$ is to set the $B$-coordinates equal to zero, and leave the $A$ coordinates untouched. Every projection is of this form, and knowing $A$ and $B$ completely characterizes the projection. A projection is orthogonal iff $B = A^\perp$.

It is an exercise in the algebra of inner products to show that if $P = \rho_{A,B}$, then $P^T = \rho_{B^{\perp}, A^{\perp}}$ (no matter if $P$ is orthogonal or not). So if $P$ is an orthogonal projection and $A = B^{\perp}$, we can see that $\rho_{B^{\perp}, A^{\perp}} = \rho_{A,B}$, and $P$ is symmetric.

Answer 3

You can also directly verify this by definition of self-adjointness:

Let $U$ be a subspace of $V$, let $P$ be the orthogonal projection that projects onto $U$; let $x, y \in V$, and we can decompose them as $x=u_x+v_x, y=u_y+v_y$, with $u_x, u_y \in U, v_x, v_y \in U^\perp$, so $\langle u_x, v_y \rangle = \langle v_x, u_y \rangle = 0 $. By definition of $P$, $P x = u_x, P y = u_y$.

Then $$ \langle Px, y \rangle = \langle u_x,u_y+v_y \rangle = \langle u_x,u_y \rangle = \langle u_x + v_x, u_y \rangle = \langle x, Py \rangle $$ it follows $P$ is self-adjoint and its matrix is Hermitian (i.e. symmetric when $V$ is real).

UPDATE: I just read your question again; here's a more intuitive explanation.

Remember:

1. $\forall A \in \mathbb{R}^{n\times n} $, $\operatorname{null}(A)$ and $\operatorname{row}(A)$ are always orthogonal complements of each other in $\mathbb{R}^n$
2. when $A$ describes a projection, $A^2=A$, $\mathbb{R}^n = \operatorname{null}(A) \oplus \operatorname{col}(A)$

Now when $A$ describes an orthogonal projection, we also have $\operatorname{null}(A) \perp \operatorname{col}(A)$, so $\operatorname{null}(A)$ and $\operatorname{col}(A)$ are orthogonal complements of each other in $\mathbb{R}^n$; this happens iff $A$ is symmetric, so that $\operatorname{row}(A) = \operatorname{col}(A)$.