I'm studying a orthonormality of inner product space using Kreyszig's Functional Analysis textbook. I'm having difficulty with some parts of it. I quoted the section that I have problem with and the questions are at the bottom of this post.
If we know that a given $x$ can be represented as a linear combination of some elements of an orthonormal sequence, then the orthonormality makes the actual determination of the coefficients very easy. In fact, if $(e_1, e_2, ... )$ is an orthonormal sequence in an inner product space $X$ and we have $x \in span$ $\{e_1. ... , e_n\}$, where $n$ is fixed, then $$ x = \sum_{k=1}^n \alpha_k e_k, \tag6$$
and if we take the inner product by a fixed $e_j$, we obtain
$$ \langle x,e_j \rangle = \langle\sum \alpha_k e_k,e_j \rangle = \sum \alpha_k \langle e_k,e_j \rangle = \alpha_j, $$
With these coefficients, (6) becomes
$$ x = \sum_{k=1}^n \langle x,e_k \rangle e_k. \tag 7$$ This shows that the determination of the unknown coefficients in (6) is simple. Another advantage of orthonormality becomes apparent if in (6) and (7) we want to add another term $\alpha_{n+1}e_{n+1}$ to take care of an
$$ \widetilde x = x + \alpha_{n+1}e_{n+1}\in \{e_1,\ldots,e_{n+1}\}; $$
then we need to calculate only one more coefficient since the other coefficients remain unchanged.
More generally, if we consider any $x \in X$, not necessarily in $Y_n = span \{e_1, \ldots ,e_n\}$, we can define $y \in Y_n$ by setting
$$ y = \sum_{k=1}^n \langle x,e_k \rangle e_k, \tag {8a}$$ where $n$ is fixed, as before, and then define $z$ by setting
$$x = y + z, \tag {8b}$$
that is, $z = x - y$. We want to show that $z \perp y$. To really understand what is going on, note the following. Every $y \in Y_n$ is a linear combination
$$ y = \sum_{k=1}^n \alpha_k e_k. $$
Here $ \alpha_k= \langle y,e_k \rangle $, as follows from what we discussed right before. Our claim is that for the particular choice $ \alpha_k= \langle x,e_k \rangle, k = 1, \ldots, n,$ we shall obtain a $y$ such that $z = x - y \perp y$.
Questions:
1) What does this specifically mean: if we consider any $x \in X$, not necessarily in $Y_n = span \{e_1, \ldots ,e_n\}$, then (8a)? What is the relationship between $X$ and $Y_n$? If they are spanned by the same basis, then aren't they the same such that $X = Y$? Which is why it is confusing.
2) Doesn't (8a) actually mean that $ y = x$?
3) The last sentence: Our claim is that for the particular choice $ \alpha_k= \langle x,e_k \rangle, k = 1, \ldots, n,$ we shall obtain a $y$ such that $z = x - y \perp y$. How does the choice of $ \alpha_k$ imply $z = x - y \perp y$?
Thank you!!!
1) The set $\{e_1,\dots,e_n\}$ is a finite subset of the orthonormal sequence; it’s not the entire sequence. Its span $Y_n$ is thus a finite-dimensional proper subspace of the infinite-dimensional space $X$.
2) If $x$ happens to be an element of $Y_n$, then of course, $y=x$. If $x\notin Y_n$, however, there’s no linear combination of the vectors $\{e_1,\dots,e_n\}$ that can equal $x$.
3) To see this formally, compute $$\begin{align} \langle y,z\rangle &= \langle y,x-y\rangle \\ &= \langle y,x\rangle-\langle y,y\rangle \\ &= \langle\sum_{i=1}^n\alpha_ie_i,x\rangle-\sum_{i=1}^n\alpha_i^2 \\ &= \sum_{i=1}^n\alpha_i\langle e_i,x\rangle-\sum_{i=1}^n\alpha_i^2 \\ &= \sum_{i=1}^n\alpha_i^2-\sum_{i=1}^n\alpha_i^2 \\ &=0.\end{align}$$ For an informal explanation, see below.
You can visualize what’s going on here quite easily in $X=\mathbb R^3$. An orthonormal sequence in $\mathbb R^3$ is necessarily finite, but that doesn’t make any fundamental difference. A subset of such a sequence $\{e_1,e_2\}$ spans some plane $Y_2$ through the origin. If take $x\notin Y_2$, we can decompose it into a component $y$ that lies within the plane and a component $z$ that’s outside of it. The in-plane component $y=\langle x,e_1\rangle e_1+\langle x,e_2\rangle e_2$ is just the orthogonal projection of $x$ onto $Y_2$. The other component, $z$, is what’s left over after you subtract this from $x$. This is the orthogonal rejection of $x$ from $Y_2$ and must be orthogonal to this plane. If it weren’t, at least one of $\langle z,e_1\rangle$ or $\langle z,e_2\rangle$ would be non-zero, but those components of $x$ have already been accounted for in $y$.