I need help with a proof showing $\|u\|^2 = \|\operatorname{proj}_v u\|^2 + \|u - \operatorname{proj}_v u\|^2 $

Question

So, I am dealing with the 2-norm and the projection is defined as the standard orthogonal projection, so far I have

$$\|u\|^2 = \|\operatorname{proj}_v u\|^2 + \|u - \operatorname{proj}_v u\|^2 = 2\|\operatorname{proj}_v u\|^2 + \|u\|^2 - 2<u,\operatorname{proj}_v u>.$$

This then gives me $\|\operatorname{proj}_v u\|^2- <u,\operatorname{proj}_v u>$ = 0, however this would mean that $u= \operatorname{proj}_v u$, which is not true. Help is appreciated, thanks in advance!

Answer 1

First of all a change notation for ease of reading. Let $P=\operatorname{proj}_v$.

Now on to your problem. If I understand what you are asking correctly, your misunderstanding comes from an incorrect implication. $\|Pu\|^2- \langle u,Pu\rangle=0$ does not imply that $u=Pu$. In fact we have $$\begin{align*}\|Pu\|^2&=\langle u,Pu\rangle \\ \langle Pu,Pu\rangle&=\langle u,Pu\rangle \quad (1) \end{align*}$$ and while it is tempting to equate $Pu$ and $u$ this is incorrect. This is perhaps illustrated best with an example. Consider the projection onto the x-axis given by the matrix $$P=\begin{bmatrix}1&0&0\\0&0&0\\0&0&0\end{bmatrix}$$ and let $u=[1,2,3]^T$.

Computing $Pu$ we get $$Pu=\begin{bmatrix}1&0&0\\0&0&0\\0&0&0\end{bmatrix}\begin{bmatrix}1\\2\\3\end{bmatrix}=\begin{bmatrix}1\\0\\0\end{bmatrix}$$ Therefore, the left hand side of $(1)$ is $$\langle Pu,Pu\rangle=\begin{bmatrix}1&0&0\end{bmatrix}\begin{bmatrix}1\\0\\0\end{bmatrix}=1\cdot1+0\cdot0+0\cdot0=1$$ and similarly the right hand side reduces to $$\langle u,Pu\rangle=\begin{bmatrix}1&2&3\end{bmatrix}\begin{bmatrix}1\\0\\0\end{bmatrix}=1\cdot1+2\cdot0+3\cdot0=1$$ Clearly $\langle Pu,Pu\rangle=\langle u,Pu\rangle$, however $Pu\ne u$, $$\begin{bmatrix}1\\0\\0\end{bmatrix}\ne\begin{bmatrix}1\\2\\3\end{bmatrix}$$ In general, if you have $\langle v,u\rangle=\langle w,u\rangle$ the best you can say about $v$ and $w$ is that they lie in the same hyperplane.

Edit: some elaboration on my closing remarks

Consider $v,w,u\in\mathbb{R}^3$ and let $u=[a,b,c]^T$ and $v$ be some vector $v=[x,y,z]^T$.

Now, $\langle w,u\rangle$ is some scalar quantity, say $d$. Therefore we have $$\langle v,u\rangle=\langle w,u\rangle\implies ax+by+cz=d$$ which defines a plane in $\mathbb{R}^3$. Any $v$ in that plane satisfies $\langle v,u\rangle=\langle w,u\rangle$ and so you cannot make the conclusion that $v=w$

Answer 2

I think $\operatorname{proj}_v u$ and $u-\operatorname{proj}_v u$ are orthogonal, so this is just the Pythagorean theorem, right?

Answer 3

Hint: Refer to the Orthogonal Decomposition Theorem.

Answer 4

How about this?

\begin{align} \| u \|^2 &= \| u - \operatorname{proj}_v u + \operatorname{proj}_v u\|^2 \\ &= \left< u - \operatorname{proj}_v u + \operatorname{proj}_v u, u - \operatorname{proj}_v u + \operatorname{proj}_v u\right > \\ &= \|u - \operatorname{proj}_v u \|^2 + \|\operatorname{proj}_v u \|^2 + \underbrace{\left < u - \operatorname{proj}_v u ,\operatorname{proj}_v u \right >}_{\text{0}} + \underbrace{\left < \operatorname{proj}_v u, u - \operatorname{proj}_v u \right >}_{\text{0}}\\ &=\|u - \operatorname{proj}_v u \|^2 + \|\operatorname{proj}_v u \|^2 \end{align}