Prob. 6, Sec. 3.4 in Kreyszig's functional analysis book: The fourier coefficients minimise the distance.

Question

Let $n \in \mathbb{N}$, let $\{ e_1, \ldots, e_n \}$ be an orthonormal set in an inner product space $X$, let $x \in X$, let $y(x) \colon= \sum_{j=1}^n \langle x, e_j \rangle e_j$, and let $z \colon= \sum_{j=1}^n \gamma_j e_j$, where $\gamma_1, \ldots, \gamma_n$ are any scalars. Then how to show that $$\Vert x-y(x) \Vert \leq \Vert x-z \Vert,$$ where $\Vert \cdot \Vert$ is the norm induced by the inner product $\langle \cdot \rangle$?

My effort:

\begin{align} \Vert x-z \Vert^2 - \Vert x-y(x) \Vert^2 &= \langle x-z, \ x-z \rangle - \langle x-y(x), \ x-y(x) \rangle \\ &= - 2 \Re \langle x, z \rangle + \Vert z \Vert^2 + 2 \Re \langle x, y(x) \rangle - \Vert y(x) \Vert^2 \\ &= - 2 \Re \sum_{j=1}^n \langle x, e_j \rangle \overline{\gamma_j} + \sum_{j=1}^n \vert \gamma_j \vert^2 + 2 \sum_{j=1}^n \vert \langle x, e_j \rangle \vert^2 - \sum_{j=1}^n \vert \langle x, e_j \rangle \vert^2 \\ &= - 2 \Re \sum_{j=1}^n \langle x, e_j \rangle \overline{\gamma_j} + \sum_{j=1}^n \vert \gamma_j \vert^2 + \sum_{j=1}^n \vert \langle x, e_j \rangle \vert^2 \\ &\geq - 2 \left\vert \sum_{j=1}^n \langle x, e_j \rangle \overline{\gamma_j} \right\vert + \sum_{j=1}^n \vert \gamma_j \vert^2 + \sum_{j=1}^n \vert \langle x, e_j \rangle \vert^2 \\ &\geq - 2 \sum_{j=1}^n \left\vert \langle x, e_j \rangle \overline{\gamma_j} \right\vert + \sum_{j=1}^n \vert \gamma_j \vert^2 + \sum_{j=1}^n \vert \langle x, e_j \rangle \vert^2 \\ &\geq - 2 \sqrt{ \sum_{j=1}^n \left\vert \langle x, e_j \rangle \right\vert^2} \cdot \sqrt{\sum_{j=1}^n \left\vert \overline{\gamma_j} \right\vert^2} + \sum_{j=1}^n \vert \gamma_j \vert^2 + \sum_{j=1}^n \vert \langle x, e_j \rangle \vert^2 \\ &= - 2 \sqrt{ \sum_{j=1}^n \left\vert \langle x, e_j \rangle \right\vert^2} \cdot \sqrt{\sum_{j=1}^n \left\vert \gamma_j \right\vert^2} + \sum_{j=1}^n \vert \gamma_j \vert^2 + \sum_{j=1}^n \vert \langle x, e_j \rangle \vert^2 \\ &= \left(\sqrt{ \sum_{j=1}^n \left\vert \langle x, e_j \rangle \right\vert^2} - \sqrt{\sum_{j=1}^n \left\vert \gamma_j \right\vert^2} \right)^2 \\ &\geq 0. \end{align}

Is this calculation correct? If not, then where does the problem lie?

Answer 1

Notice that $$ \left(x-\sum_{k=1}^{n}\langle x,e_k\rangle e_k\right) \perp e_j,\;\;\; j=1,2,3,\cdots, n. $$ Therefore, for all scalars $\beta_1,\beta_2,\cdots,\beta_n$, $$ \left(x-\sum_{k=1}^{n}\langle x,e_k\rangle e_k\right)\perp\sum_{j=1}^{n}\beta_j e_j $$ Therefore, by the Pythagorean Theorem, \begin{align} \left\|x-\sum_{j=1}^{n}\alpha_je_j\right\|^2&= \left\|\left(x-\sum_{j=1}^{n}\langle x,e_j\rangle e_j\right)+\sum_{j=1}^{n}(\langle x,e_j\rangle-\alpha_j)e_j\right\|^2 \\ & = \left\|x-\sum_{j=1}^{n}\langle x,e_j\rangle e_j\right\|^2+\left\|\sum_{j=1}^{n}(\langle x,e_j\rangle-\alpha_j)e_j\right\|^2 \end{align} Now you can see that the following holds for all choices of $\alpha_j$ $$ \left\|x-\sum_{j=1}^{n}\alpha_je_j\right\| \ge \left\|x-\sum_{j=1}^{n}\langle x,e_j\rangle e_j\right\|, $$ and you have equality iff $\alpha_j = \langle x,e_j\rangle$ for all $j$.

Answer 2

Your calculation looks good. There is, however, an easier solution. First, it is not hard to prove that the law of cosines also holds in arbitrary Hilbert spaces: $$\|x-z\|^2 = \|x - y\|^2 + \| z - y\|^2 - 2 \, \Re \langle z - y , x - y\rangle$$ for all $x,y,z$ in a Hilbert space. This relation is also convenient to use in other situations.

In your situation, you have $\langle z - y(x), x - y(x)\rangle = 0$ due to the construction of $y(x)$. Hence, $$\|x - z\|^2 = \|x - y(x)\|^2 + \|z - y(x)\|^2 \ge \|x - y(x)\|^2.$$ This method has the advantage that you can precisely characterize the difference of the (squared) norms, since $$\|z - y(x)\|^2 = \sum_{j = 1}^n |\gamma_j - \langle x,e_j\rangle|^2.$$

(by the way: this can also be seen from your fourth line, so you can short-cut your proof.)