Prove that for any nonzero vectors $\bf u$ and $\bf a$ in $\Bbb R^n$, the vector $\bf a$ is orthogonal to ${\bf u} - \mathrm{proj}_{\bf a}{\bf u}$.

Question

Prove that for any nonzero vectors $\bf u$ and $\bf a$ in $\Bbb R^n$, the vector $\bf a$ is orthogonal to ${\bf u} - \mathrm{proj}_{\bf a}{\bf u}$.

I'm not sure how to start proving this. I don't think I've seen any proofs involving projections at all, just the theorems describing what they are... I've been studying my projection notes and graph examples non stop trying to visualize a way to prove it...

Answer 1

Let $\langle \cdot , \cdot \rangle$ denote the standard inner product in $\mathbb{R}^n$. Let $a,u$ be nonzero elements of $\mathbb{R}^n$. The projection of $u$ onto $a$ is given by $$\mathrm{proj}_a(u)=\frac{\langle a, u\rangle}{|a|^2}a.$$

By definition, $a$ is orthogonal to $u-\mathrm{proj}_a(u)$ if we have $\langle a,u-\mathrm{proj}_a(u)\rangle=0$. We compute this inner product: \begin{align*} \langle a,u-\mathrm{proj}_a(u)\rangle&=\langle a, u\rangle -\langle a,\mathrm{proj}_a(u)\rangle\\ &=\langle a, u\rangle -\langle a, \frac{\langle a, u\rangle}{|a|^2}a \rangle \\ &=\langle a, u\rangle-\frac{\langle a, u\rangle}{|a|^2}\langle a, a\rangle \\ &=\langle a, u\rangle-\frac{\langle a, u\rangle}{|a|^2}|a|^2\\ &=\langle a, u\rangle-\langle a, u\rangle=0. \end{align*} Thus $a$ is orthogonal to $u-\mathrm{proj}_a(u)$, as desired. Here the only thing I have used is the bilinearity of the dot product and the definition of $|\cdot |$.

Answer 2

Hint: Take the dot product of $\vec{a}$ with $\displaystyle\vec{u}-\text{proj}_{\vec{a}}\vec{u}=\vec{u}-\frac{\vec{a}\cdot\vec{u}}{\vec{a}\cdot\vec{a}}\vec{a}$$\;\;$ and show that it is zero.

Answer 3

You want to check that $$\langle {\bf a}, {\bf u} - \mathrm{proj}_{\bf a} {\bf u} \rangle = 0. $$

Remember that, by definition, we have: $$\mathrm{proj}_{\bf a}{\bf u} = \frac{\langle {\bf a}, {\bf u} \rangle}{\langle {\bf a},{\bf a}\rangle} {\bf a}.$$

Substitute and use that $\langle \cdot , \cdot \rangle $ is bilinear. Can you do it?

Answer 4

An alternative definition states: $$ \text{proj}_V u = \text{argmin}_{v\in V} \| v-u\|^2 $$ Now with the element $x\in v$ written $v = x a, x\in\Bbb R$: $$ \text{proj}_V u = \text{argmin}_{x\in \Bbb R} \| xa-u\|^2 \times a $$and the critical points are solution of: $$ 0 = \frac{d}{dx} \| xa-u\|^2 = 2\langle a, xa-u \rangle = 2\langle a, \text{proj}_V u-u \rangle $$