I'm having trouble understand a big step on the proof of a theorem in this book I'm reading. The authors provide a free PDF here and the part I'm stuck in is in page 52 of the book and involves some clever trick to open up a squared norm.
The relevant part of the theorem reads:
Theorem 2.26 (Projection theorem) Let $S$ be a closed subspace of a Hilbert space $H$, and let $x$ be a vector in $H$.
(ii) Orthogonality: $x − \hat x \perp S$ is necessary and sufficient for determining $\hat x$.
I'm reproducing the proof here up to the step I can't understand, which I mark as $=^b$:
(ii) Orthogonality: Suppose that $\hat x$ minimizes $\lVert x − \hat x \rVert$ but that $x − \hat x \not\perp S$ . Then, there exists a unit vector $\varphi \in S$ such that $\langle x - \hat x, \varphi \rangle = \epsilon \neq 0$. Let $s = \hat x + \epsilon\varphi$ and note that $s$ is in $S$ since $x$ and $\varphi$ are in $S$ and $S$ is a subspace. The calculation:
$$ \lVert x - s \rVert ^ 2 = \lVert x - \hat x - \epsilon\varphi \rVert ^ 2 =^b \lVert x - \hat x \rVert ^ 2 - \langle x - \hat x, \epsilon\varphi \rangle - \langle \epsilon\varphi, x - \hat x \rangle + \lVert \epsilon\varphi \rVert ^2 = ... $$
The proof continues by noting the last three terms are equal to $\vert \epsilon \vert ^2 $ and it simplifies easily. I'm not copying the rest of the proof because I understand it, but this step is a bit much. I tried looking into all the internal product and norm properties in the book but I think I'm missing something related to how they define $\varphi$.
I also googled different proofs for this theorem, and the only one that states it in a similar manner uses this same proof and there's also no explanation for this step. Hope someone can help me here!
Thanks!
It follows from linearity of the inner product: $$\|x-\hat{x}-\epsilon \varphi\|^2=\langle x-\hat{x}-\epsilon \varphi,x-\hat{x}-\epsilon \varphi\rangle=$$ $$=\langle x-\hat{x}, x-\hat{x} - \epsilon \varphi\rangle+\langle -\epsilon \varphi,x- \hat{x} - \epsilon \varphi\rangle =$$ $$=\langle x-\hat{x}, x-\hat{x}\rangle+\langle x-\hat{x},- \epsilon \varphi\rangle+ \langle - \epsilon \varphi, x-\hat{x}\rangle+\langle -\epsilon \varphi, -\epsilon \varphi\rangle=$$ $$=\|x-\hat{x}\|^2-\langle x-\hat{x}, \epsilon \varphi\rangle- \langle \epsilon \varphi, x-\hat{x}\rangle+\|\epsilon \varphi\|^2$$