Coefficients in Generalised Fourier series Given By $c_k = \langle f, \psi_k \rangle$? $c_k$ Implicitly Chosen So That Sum Converges?

Question

My notes state the following:

The coefficients $c_k$ in the generalised Fourier series

$$f(x) = \sum\limits_{k = 1}^\infty c_k \phi_k(x)$$

, with respect to the orthonormal set, $\{ \psi_k\}$,

$$f(x) = \sum\limits_{k = 0}^\infty c_k \psi_k(x)$$

, are given by $c_k = \langle f, \psi_k \rangle$.

Note: We have implicitly assumed that it is possible to choose the coefficients $c_k$ so that the sum

$$\sum\limits_{k = 0}^N c_k \psi_k(x)$$

converges to $f(x)$ as $N \to \infty$.

I have two questions about this:

1. Why are the coefficients $c_k$ in the generalised Fourier series, with respect to the orthonormal set, given by $c_k = \langle f, \psi_k \rangle$? Can someone please demonstrate this?

2. How is it that we have implicitly assumed that it is possible to choose the coefficient $c_k$ so that the sum converges to $f(x)$ as $N \to \infty$? I don't see why this is the case.

I would greatly appreciate it if people could please take the time to clarify this.

Answer 1

The statement about convergence is vague. But here is the basic idea: Suppose $\{\psi_k\}$ is orthonormal on $[a,b],$ and there are constants $c_k$ such that

$$f(x) =\sum_{k=1}^{\infty}c_k\psi_k(x),\,\,x\in [a,b].$$

Then the $c_k$ are given by the formula

$$c_k = \int_a^b f(x)\psi_k(x)\,dx, \,\,k=1,2,\dots.$$

Proof: Fix $k=k_0.$ Then

$$\int_a^b f(x)\psi_{k_0}(x)\,dx = \int_a^b \left (\sum_{k=1}^{\infty}c_k\psi_k(x)\right )\psi_{k_0}(x)\,dx$$ $$ =\int_a^b \left (\sum_{k=1}^{\infty}c_k\psi_k(x)\psi_{k_0}(x)\right)\,dx = \sum_{k=1}^{\infty}c_k\int_a^b \psi_k(x)\psi_{k_0}(x)\,dx$$ $$ = c_{k_0}\int_a^b \psi_{k_0}(x)\psi_{k_0}(x)\,dx = c_{k_0}\cdot 1 = c_{k_0}.$$

If you've never seen this kind of argument before, the first expression may look like it's pulled out of a hat. But it's the thing that works. We get to the second expression by simple substitution for $f.$ The third expression should explain itself. In the fourth expression we have switched summation and integration. This needs justification, but with your permission we just sail on through. The rest of the expressions come about by orthonormality: Those integrals equal $0$ except when $k=k_0,$ where it equals $1.$

I'm hoping this answers question 1. For question 2. the answer is simple: If we don't have coefficients $c_k$ such that $\sum c_k\psi_k =f$ in some manner, then how could we substitute $\sum c_k\psi_k$ in for $f$ in the second integral?

Answer 2

If the orthonormal set is complete then the $c_k$'s exist for any square integrable function $f$. Otherwise there exist square integrable functions for which the coefficients do not exist. If the orthonormal set is complete then the expansion $f=\sum c_k \psi_k$ is valid in $L^{2}$ norm. This means that if $S_n=\sum_{k=1}^{n} c_k \psi_k$ then $\int |S_n-f|^{2} \to 0$. This implies $\int f(x) \psi _j (x)\, dx =\lim \int f(x) S_n(x)\, dx$ Using orhtonormality we see that $\int f(x) s_n(x)\, dx=c_j$ for all $n \geq j$ which gives the formula for $c_j$.

Answer 3

For your second question, you just need to recall the definition of convergence for series:

Given the series $\sum_{k=0}^\infty v_k$, define the sequence $s_N$ of partial sums as $s_N=\sum_{k=0}^N v_k$. Then, it is said that the series converges to $S$, or that $\sum_{k=0}^\infty v_k=S$ if $s_N\to S$.

So, if you say that $f(x)=\sum_{k=0}^\infty c_k\psi_k$, by definition you're saying that the sequence $s_N=\sum_{k=0}^N c_k\psi_k$ converges to $f(x)$. Then, you're assuming the existence of such $c_k$'s.

To answer your first question, you just need to use the linearity of the inner product:

$$(f,\psi_k)=(\sum_{n=0}^\infty c_n\psi_n,\psi_k)=\sum_{n=0}^\infty c_n(\psi_n,\psi_k)=c_k,$$ where the last equation is a consequence of the set $\{\psi_k\}$ being orthonormal, since $(\psi_n,\psi_k)=0$ if $n\neq k$ and $(\psi_k,\psi_k)=1$.

Answer 4

When defining an orthonormal set $\{\psi_i(x)\}$ we have that$$\int_{D}\psi_i(x)\psi_j^*(x)dx=\begin{cases}1&,i=j\\0&,i\ne j\end{cases}$$where $D$ is the common domain of all $\{\psi_i(x)\}$ (take it $\Bbb R$) and $*$ operator denotes the complex conjugate. Using this definition $$f(x)=\sum_{n=1}^{\infty}c_n\psi_n(x)$$therefore $$\int_{D}f(x)\psi_k^*(x)dx=\sum_{n=1}^{\infty}c_n\int_{D}\psi_n(x)\psi_k^*(x)dx=\sum_{n=1}^{\infty}c_n\delta_{n-k}=c_k$$which completes the proof of question 1.

The answer to the question 2 is a little hard and tricky. As far as I know there are no known necessary and sufficient conditions to ensure existence of such coefficients neither in classical Fourier Series nor in the general case (for example in the case of Fourier Series we may choose some non-Reimann integrable periodic function such as Dirichlet function), but we have some sufficient conditions called Dirichlet conditions after a German mathematician Peter Gustav Lejeune Dirichlet. You may refer to it in https://en.wikipedia.org/wiki/Dirichlet_conditions