Is the sum of an infinite series of elements in the span of an orthonormal set also in that set?

Question

If $(e_k)$ is an orthonormal sequence in some Hilbert space $H$ does it follow that, if for a set of scalars $\{\alpha_k\}$, the series $$\sum_{k=1}^{\infty}\alpha_ke_k$$ converges to an $x \in H$, then $x\in span(e_k)=M$? For a finite sum the answer would obviously be true, but I often miss subtleties when infinity comes to play. I think that we could possibly have $x\in \bar M$ and not $x\in M$ if there are an infinite number of non-zero scalars in $\{\alpha_k\}$, but before pursuing this I'd like to confirm whether $x$ will always just be an element of the span or not.

EDIT: The two excellent answers below indicate to me that I have asked a question with answers possibly above my needs and current ability to understand. So to hopefully aid answering I'd like to give more context to my question. The question stems from a problem in Kreyszig's introductory functional analysis textbook, which, in the terminology developed above, asks: "For any $x\in H$ show that $x\in \bar M$ if $x=\sum_{k=1}^{\infty}\langle x,e_k\rangle e_k$". If the solution to this problem is different from the general question I asked please explain why and if possible give a hint for a possible proof I could use. I should also add that I am not at all familiar with topology as that course follows functional analysis in my university.

Answer 1

Kreyszig Problem: For any $x\in H$ show that $x\in \bar M$ if $x=\sum_{k=1}^{\infty}\langle x,e_k\rangle e_k$. (It is assumed that $M$ means all finite linear combinations of the orthonormal set $\{ e_k \}_{k=1}^{\infty}$.)

Let $M_{n}$ be the subspace spanned by $\{ e_1,e_2,\cdots,e_n \}$, and let $x \in H$ be given. The closest point $m_n \in M_n$ to $x$ is given by the orthogonal expansion $m_n = \sum_{k=1}^{n}\langle x,e_k\rangle e_k\in M_n$. That is, $$ \|x-\sum_{k=1}^{n}\langle x,e_k\rangle e_k\| = \inf_{y\in M_n}\|x-y\|. $$ Because of this, $\|x-m_1\| \ge \|x-m_2\| \ge \cdots \ge \|x-m_n\| \ge \cdots$. Suppose $x \in \overline{M}$. Then, for every $\epsilon > 0$, there exists $x_N$ in some $M_N$ such that $\|x-x_N\| < \epsilon$. Hence, $$ \|x-\sum_{k=1}^{n}\langle x,e_k\rangle e_k \| \le \|x-x_N\| < \epsilon,\;\;\; n \ge N. $$ Therefore $x\in \overline{M} \implies \lim_{N\rightarrow\infty}\|x-\sum_{n=1}^{N}\langle x,e_k\rangle e_k\| = 0$.

On the other hand, if $\lim_{N}\|x-\sum_{k=1}^{N}\langle x,e_k\rangle e_k\|=0$, then $x$ is the limit of a sequence in $M$, which means $x\in \overline{M}$.

Answer 2

No, it is not true in general. It is true if the sequence of scalars is in $\ell^2$, or square summable.

Moreover, we need to clarify a few things in your question and thinking.

(1) Most people mean the algerbaic span when they say ``span''. This is the set of all finite linear combinations of basis elements. So, trivially, an infite linear combination will not be in the span. When you're dealing with a Hilbert Space, or any infinite-dimensional linear space, you almost always need to have some topology that allows to you talk about convergence. This is the only way you can talk about infinite sums- as limits of finite sums. This shouldn't be a crazy idea because think back to the way infinite sums were defined in Calculus II.

(2) Now, if we define an infinite sum $$ \sum_{n=1}^{\infty} a_n e_n :=\lim_{m\to \infty} \sum_{n=1}^{m} a_n e_n, $$ the question still remains for which sequence of scalars $\{a_n\}_{n=1}^{\infty}$ will the right-hand-side converge? It turns out that the answer is all square-summable sequences. If you don't understand why, look back to Parseval's indentity in your text.

(3) Now, will the limit be in the span? As explain before, no. However, if all of the orthonormal basis elements that we want to sum come from some linear space, then the infinite sum will be in the topological closure of the space. If the linear subspace was already closed, then the limit will be in the space.

The book you're using may be using the word span to mean closure, or ``Hilbert Space Span''. But in conventional language the limit is in the closure and not the span.

Hope this helps!

Answer 3

Consider the space of square-integrable functions on $[-1,1]$ and the sequence $e_k = \frac{2k+1}{2} P_k(x)$ where $P_k(x)$ is the $k$-th Legendre polynomial. These $e_k$ are orthonormal.

Now take $\alpha_k = \frac{1}{\sqrt{k}}$ and $f(x) = \sum \alpha_k e_k(x)$. Then

$$ \int_{-1}^1 [f(x)]^2 dx =\sum \frac1{k} $$

which diverges, so $f(x)$ is not in the span.

But wait! $f(x)$ is also not in the Hilbert space. SO this is not a good counterexample.

The problem is that any non-square-summable sequence $\alpha_k$ leads to a function which is outside the Hilbert space of square-integrable functions.

Thus I am uncomfortable about the first sentence in Alex's answer: If the statement is true for all square-summable sequences of $\alpha_k$ then why is it not true in general, since it specifies that the sum has to be in $H$? I thinik that the statement is in fact true.

Answer 4

Note that if $x=\sum \alpha_k e_k$, then the scalars $\alpha_k$ are uniquely determined by $x$, since $\langle x,e_\ell\rangle=\sum\alpha_k\langle e_k,e_\ell\rangle=\alpha_\ell$. In particular, if $x$ is in $M$, then by definition it must be a finite linear combination of the $e_k$, meaning that it can be written in the form $x=\sum\alpha_ke_k$ where all but finitely many $\alpha_k$ are $0$. By the uniqueness noted above, this means it is impossible to write $x$ as a sum $\sum \alpha_ke_k$ where infinitely many of the $\alpha_k$ are nonzero.

This means that if you have a convergent sum $\sum \alpha_ke_k$ for which infinitely many of the $\alpha_k$ are nonzero, its limit cannot be an element of $M$. On the other hand, such a sum always is in $\overline{M}$, simply because each finite partial sum $\sum_{k=1}^N\alpha_k e_k$ is in $M$, and these partial sums converge to the infinite sum. In particular, if $x=\sum\langle x,e_k\rangle e_k$, we conclude that $x\in\overline{M}$ (just write $\alpha_k=\langle x,e_k\rangle$).