How is $I-vv^T$ the projection onto the orthogonal complement of the line through the origin in the direction $v$?

Question

Given a vector $v\in \mathbb{R}^n$, I have seen from several sources that $I-vv^T$ is the projection onto the orthogonal complement of the line through the origin in the direction $v$?

I have unfortunately memorized this expression instead of figuring out what it actually means. I know that the orthogonal complement of a line is a plane with the line being its normal vector. And from what I understand this means that $P=I-vv^T$ is a projection matrix which places vectors into that aforementioned plane. But beyond that, I'm having difficulty connecting $I-vv^T$ to all vectors which are orthogonal to $v$ (which is what an orthogonal complement is, after all).

Can someone help me understand how this is true?

Answer 1

Given a vector $x$, and a unit vector $v$, the projection of $x$ onto $v$ is going to be $(v^Tx)v=v(v^Tx)=(vv^T)x$.

Hence the orthogonal complement is $x-(v^Tx)v=x-(vv^T)x=(I-vv^T)x.$

Answer 2

We have that

$$(vv^T)b=(v^Tb)v$$

is the othogonal projection of $b$ onto $v$ then

$$b-(vv^T)b=(I-vv^T)b$$

is the projection onto the orthogonal complement.

Answer 3

Let $v$ an unit vector and $(v,u_1,\dots,u_{n-1})$ an orthonormal basis of $\mathbb{R}^n$ (Constructed by Grahm-Schmitt for example).

Then $x$ can be decomposed in that basis, i.e. : $$x=\lambda v+\sum_{i=1}^{n-1} \lambda_i u_i.$$

Then $v^Tx$ is the inner product and its values is $\lambda$. Thus $vv^Tx= \lambda v$. In other words, $vv^Tx$ is the orthogonal projection into the vector space spanned by $v$.

Then $$(I-vv^T)x=\sum_{i=1}^{n-1} \lambda_i u_i.$$

What we have done is removing the $v$ component of $x$. Since our basis is orthogonal, $$v^T(I-vv^T)x=\sum_{i=1}^{n-1}\lambda_i v^Tu_i=0.$$

$(I-vv^T)$ fails to be a projection into $ Span(u_1,..,u_{n-1})$ if $v$ is not a unit vector, as $vv^Tx=v\|v\|_2\lambda$, hence :

$$(I-vv^T)x=(1-\|v\|_2)\lambda v +\sum_{i=1}^{n-1} \lambda_i u_i.$$

which still have a component in $v$.

Answer 4

Let $E$ be the orthogonal space to the line defined by $v$. The orthogonal projection $P: \mathbb R^n \to E$ is the unique operator such that the following conditions are satisfied:

• The image of $P$ is $E$
• $\langle x - Px, E \rangle = 0$
• $E$ is $P$-invariant

To see uniqueness, suppose that $P'$ was another orthogonal projection, then

$$ 0 = \langle Px - P'Px, E \rangle = \langle Px - P'x, E \rangle, $$ and since $P, P'$ both map to $E$, they are equal. (Existence is typically shown by using an orthonormal basis of $E$ that is extended to an orthonormal basis of $\mathbb R^n$, using e.g. basis extension and Gram--Schmidt.) Hence, it is sufficient to show that your projection satisfies these properties.

So, let $x \in \mathbb R^n$.

• $I - v v^t$ maps to $E$, since $\langle (I - vv^t)x, v \rangle = 0$ (here we need $v$ to be a unit vector and use $\langle v v^t x, v \rangle = \langle x, v v^t v \rangle$. For reasons of dimension ($vv^t$ is a matrix of rank $1$), the image equals $E$.
• This is because $\langle -v v^t x, y \rangle = 0$ for $y \in E$.
• This is again by orthogonality, namely $v v^t y = v \langle v, y\rangle = 0$ for $y \in E$.