I am reading the book Self-organizing maps by Kohonen.
In Theorem 1.1.1 he states
Of all decompositions of the form $x = x' + x''$ where $x'\in \mathcal{L}$, the one into orthogonal projections has the property that $||x''||$ is minimum.
First question is, what does he mean by "minimum"? I have learned Math in a different language from English, so I am not quite sure, what he means by that.
He goes on with the proof
To prove this theorem, use is made of the definition $||x'|| = (x',x')$ and the facts that $\hat{x} - x' \in \mathcal{L}$, $\quad x - \hat{x} = \tilde{x} \perp \mathcal{L}$, whereby $(\hat{x} - x', x - \hat{x}) = 0$. The following expansion then yields
Until here I am still understanding where he is going. Since both $\hat{x}, x' \in \mathcal{L}$ then the difference also must be $\in \mathcal{L}$. The part $\hat{x} = \tilde{x} \perp \mathcal{L}$ is basically the definition of the orthogonal projection, so I am also fine with this. And $(\hat{x} - x', x - \hat{x}) = 0$ basically means that both expressions are orthogonal to each other. $(x,y)$ is defined as the dot-product.
Now he continues with something I don't get how he gets from the second part to the third part of the equation.
$||x-x'||^2 = (x-\hat{x} + \hat{x} -x',x-\hat{x} + \hat{x} -x') = ||x-\hat{x}||^2 + ||\hat{x}-x'||^2$
He finishes the proof with
Because the squared norms are always positive or zero, there can be written $ ||x - x'||^2 \geq ||x - \hat{x}||^2 $, from which it is directly seen that $x'' = x - x'$ is minimum for $x' = \hat{x}$ whereby $x'' = \tilde{x} \quad \square$
I am not quite sure if I get the last part correctly, since I don't understand the part before correctly.
It's just the Pythagorean theorem: if $a\perp b$, then $\|a+b\|^2=\|a\|^2+\|b\|^2$ because $$(a+b,\,a+b)=(a,a)+(a,b)+(b,a)+(b,b)=(a,a)+(b,b)$$ since $(a,b)=(b,a)=0$.
Apply it for $a=x-\hat x$ and $b=\hat x-x'$.