Proving Bessel's Inequality, with equality if and only if $u \in Span(v_1,...,v_i)$

Question

Assume $(v_1,...,v_k)$ is an orthonormal set in an inner product space $(V, \langle \cdot, \cdot\rangle)$ and $u \in V$. Prove Bessel's inequality, with equality if and only if $u \in \text{Span}(v_1,...,v_k)$.

I have looked up proofs of Bessel's Inequality, but I'm struggling with proving equality if and only if $u \in \text{Span}(v_1,...,v_k)$. I made a nasty attempt at the backwards direction, and I seem to have leaps to where I want to be in the proof, but I eventually reached at the conclusion that equality holds at $0 = 0$. Any help on how I would tackle the forward direction, or any important points I should pay attention to in the backwards direction?

Answer 1

If $u \in \newcommand{\span}{\mathop{\rm span}} \span \{v_1,\ldots,v_n\}$ write $$u = \sum_{k=1}^n \lambda_k v_k.$$ For each index $j$ you get $$\langle u,v_j \rangle = \sum_{k=1}^n \lambda_k \langle v_k,v_j \rangle = \lambda_j$$ because the $\{v_k\}$ are orthonormal. Consequently $$\|u\|^2 = \langle u,u \rangle = \sum_{k=1}^n \sum_{j=1}^n \lambda_k \overline{\lambda_j} \langle v_k,v_j \rangle = \sum_{k=1}^n |\lambda_k|^2 = \sum_{k=1}^n |\langle u,v_k \rangle |^2.$$

Conversely, if equality holds you could start with the observation that if $u$ is going to belong to $\span \{v_1,\ldots,v_n\}$, its coefficients $\lambda_k$ must, as above, have the form $\lambda_j = \langle u,v_j \rangle$. So consider \begin{align*} \left\| u - \sum_{k=1}^n \langle u,v_k \rangle v_k \right\|^2 = \|u\|^2 &- \sum_{k=1}^n \overline{\langle u,v_k \rangle} \langle u,v_k \rangle - \sum_{k=1}^n \langle u,v_k \rangle \langle v_k,u \rangle \\ &+ \sum_{k=1}^n \sum_{j=1}^n \langle u,v_k \rangle \overline{\langle u,v_j \rangle} \langle v_k,v_j \rangle \end{align*} This reduces (again using orthonormality) to $$\|u\|^2 - \sum_{k=1}^n |\langle u,v_k \rangle|^2 = 0$$ so that $$u = \sum_{k=1}^n \langle u,v_k \rangle v_k.$$

Answer 2

We have

\begin{align} 0 &\le \left\|u - \sum_{i=1}^k \alpha_i v_i\right\|^2\\ &= \left\langle u - \sum_{i=1}^k \alpha_i v_i , u - \sum_{i=1}^k \alpha_i v_i\right\rangle\\ &= \|u\|^2 + \sum_{i=1}^k \Big(-\overline{\alpha_i}\langle u, v_i\rangle - \alpha_i \overline{\langle u, v_i\rangle} + |\alpha_i|^2\Big)\\ &= \|u\|^2 - \sum_{i=1}^k \left|\langle u, v_i\rangle\right|^2 + \underbrace{\sum_{i=1}^k \left|\alpha_i - \langle u, v_i\rangle\right|^2}_{\ge 0} \end{align}

Therefore $\sum_{i=1}^k \left|\langle u, v_i\rangle\right|^2 \le \|u\|^2$.

In particular, for $\sum_{i=1}^k \langle u, v_i\rangle$ we have

$$\left\|u - \sum_{i=1}^k \langle u, v_i\rangle v_i\right\|^2 = \|u\|^2 - \sum_{i=1}^k \left|\langle u, v_i\rangle\right|^2$$

Hence, the equality $\sum_{i=1}^k \left|\langle u, v_i\rangle\right|^2 = \|u\|^2$ holds if and only if $u = \sum_{i=1}^k \langle u, v_i\rangle v_i$ which holds if and only if $v \in \operatorname{span}\{v_1, \ldots, v_k\}$.