The link between vectors spaces ($L^2(-\pi, \pi$) and fourier series

Question

So in my PDE course we started with a review of complex numbers and vector spaces to introduce us to fourier series. I have a few questions about this.

1. I know 'big ell 2' and 'little el 2' are vector spaces. However I need a little bit more understanding on what these are. The lecture defined them ast $$l^2 := e^n(m) = \begin{cases} 0 & \mbox{if } m \neq n \\ 1 & \mbox{if } m = 1 \end{cases} $$ and $$ L^2 := V=\{ f_x : [-\pi, \pi] \rightarrow \mathbb{C} | \frac{1}{2\pi}\int_{-\pi}^\pi {|f_x|^2 dx} < \infty \}$$. Can you explain more about these vector spaces, why they are important, a wiki page, and basis why/how the basis are orthonormal and infinite dimensional.
2. Since they(what is "they") are orthonormal $f_x$ from $L^2$ wraps "something" around the unit circle.$f_x $ is defined as $f_n(x) = e^{inx}$ and for every x, $|e^{inx}| = 1$
3. Enough about them, lecture notes says Fourier series connects $L^2([-\pi,\pi])$ to $l^2(\mathbb{Z})$ using orthognal projection. I am very confused about this.

The reason I am asking here is that I can't find a good resource (pdf, wiki) that conects linear algebra to fourier series. Theres plenty of information on either subject but not for both.

Answer 1

I recommend reading the chapter on Elementary Hilbert Space Theory in Rudin's Real and Complex Analysis if you want a full answer. However, the basic ideas are the following.

$$L^2(-\pi, \pi) = \{f: (-\pi, \pi) \to \mathbb{C}: \int_{-\pi}^\pi |f(x)|^2 dx < \infty\}$$ This is a vector space if you take pointwise addition and scalar multiplication. It is a Hilbert space if you introduce the inner product $$\langle f, g \rangle = \int_{-\pi}^\pi f(x) \bar{g(x)} dx$$

The space $l^2$ is the set of complex sequences whose squares are summable: $$l^2 = \{(a_n): \sum_{n \in \mathbb{Z}} |a_n|^2 < \infty\}$$

It turns out (this is needs a simple but rather lengthy proof) that the functions $\{e^{inx}: n \in \mathbb{Z}\}$ are an orthonormal basis of the Hilbert space $L^2(-\pi,\pi)$, in the sense that for each $f \in L^2$ there exists a unique sequence of complex numbers $\{c_n: n \in \mathbb{Z}\}$ such that $$f = \sum_{n \in \mathbb{Z}} c_ne^{inx}$$ and that $\langle e^{inx}, e^{imx} \rangle = 1$ if $m = n$ and $0$ otherwise. This latter part is what it means to be orthonormal. It is important to notice that the limit in this sum is taken in the $L^2$ norm. The sum does NOT necessarily converge pointwise or in other norms. It also turns out that the sequences $(c_n)$ found in this way is in $l^2$, and that any sequence in $l^2$ yields an $L^2$ function by the above sum. The sequences $(c_n)$ is called the Fourier series of $f$ and can be computed using the formula $$c_n = \langle f, e^{inx}\rangle = \int_{-\pi}^\pi f(x)e^{-inx} dx$$

So you can think of the Fourier series as giving you the coordinates of a function in $L^2$ with respect to the orthonormal basis $\{e^{inx}\}$. This is what is meant by saying that the $n$-th partial sum of the Fourier series is the projection onto the subspace generated by the first $n$ basis functions.

That's a very quick summary. Again, I recommend Rudin's book for a full explanation of whats going on here. There are a lot of details to pay attention to.

Answer 2

(I'm just going to try and answer the third question for now; if you clarify the other ones at some point, I might add to this.)

As you know, $L^2[-\pi,\pi]$ is a vector space and it carries with it an inner product, defined as $\langle f,g\rangle=\frac{1}{2\pi}\int_{-\pi}^\pi f\;\bar{g}\,dx$. You have also identified some special vectors $f_n(x)=e^{inx}$ in your vector space (note that "vectors" are now functions; this might be confusing at first). You can calculate by hand that these vectors are orthonormal, that is that $$\langle f_n,f_m\rangle=\begin{cases}0;&n\neq m\\1;&n=m\end{cases}$$ You can think of this situation as having a bunch of perpendicular vectors in $\mathbb{R}^n$, if you're more familiar with that.

Now the tricky part: in $\mathbb{R}^n$, having $n$ mutually perpendicular vectors would mean that these $n$ vectors are a basis for $\mathbb{R}^n$, so you could express any vector in $\mathbb{R}^n$ as a finite linear combination of them. As it turns out, we cannot have a finite (or even countably infinite) set of vectors in $L^2[-\pi,\pi]$, such that the same would hold. What we do have, however, are the vectors $f_n$.

It so happens that we can express any vector (function) $f$ in $L^2[-\pi,\pi]$ as an infinite linear combination of the $f_n$ (interpreted in an appropriate sense). This infinite linear combination is the Fourier series for $f$. It can be shown that for any $f\in L^2[-\pi,\pi]$, the sequence of the coefficients of the Fourier series lies in $\ell^2$, and also that any sequence in $\ell^2$ gives, via a Fourier series, a function in $L^2[-\pi,\pi]$. What exactly this has to do with orthogonal projection is hidden in the proofs of these statements.