Coefficients in Fourier series

Question

This is my first post so please go easy, I don't know all the rules yet.

I was reviewing Fourier series for the 5th time and I realized that every explanation I read goes into the orthogonality of Sin and Cosine with some integrals and jumps into the formulae for determining the coefficients.

I don't understand how those formulae work to magically produce the coefficients and how this is related to the orthogonality of Sin and Cosine. I'm giving Fourier series a last shot for intuitive understanding before I accept it as fact and rote learn the formulae, please help!

Thank you

Answer 1

It's enlightening to think about an analogous idea in linear algebra. Suppose $\{v_1,v_2,\ldots,v_N \}$ is an orthonormal set of vectors in $\mathbb R^N$. Suppose also that $x \in \mathbb R^N$, and we want to write $x$ as a linear combination of the vectors $v_1,\ldots, v_N$: \begin{equation} x = \sum_{i=1}^N c_i v_i. \end{equation} How can we determine the coefficients in this linear combination? A trick that lets you determine the coefficients very easily is to take the inner product of both sides with $v_j$: \begin{align} \langle x, v_j \rangle &= \left \langle \sum_{i=1}^N c_i v_i, v_j \right \rangle \\ &= \sum_{i=1}^N c_i \langle v_i, v_j \rangle \\ &= c_j. \end{align} (Notice that in the last step, all but one of the terms vanishes, which is awesome.)

So we have discovered that \begin{equation*} c_j = \langle x, v_j \rangle. \end{equation*}

This great trick is called the "Fourier trick". (At least, that's what David Griffiths calls it in his physics books.) The same trick allows you to easily compute the coefficients in a Fourier series.

Answer 2

Inner-product spaces grew out of generalizing this trick of Fourier. So using inner product spaces to make sense of Fourier's trick reverses time History.

Fourier was not the first to discover this integral "orthogonality" property of the trigonometric functions on $[0,2\pi]$: $$ 1,\cos(x),\sin(x),\cos(2x),\sin(2x),\cdots $$ The trigonometric expansion problem had been posed much earlier in regard to the vibrational modes of a string. A complete solution of the vibrating string problem could be obtained in terms trigonometric functions in time and space, but only if one could find a way to match the initial data (t=0) to the given, initial displacement function $f$. That is, there was a problem in determining constants $a_{n}$, $b_{n}$ such that $$ f(x) = a_{0}+a_{1}\cos(x)+b_{1}\sin(x)+a_{2}\cos(2x)+b_{2}\sin(2x)+\cdots. $$ The problem seemed intractable at the time because there were infinitely many unknowns. However, assuming the above held for some such constants, Euler and Clairaut noticed that one could multiply by one of the $\sin$ or $\cos$ terms and integrate over the full period $[0,2\pi]$, and all the terms on the right would be $0$ except for the one involving the same $\sin$ or $\cos$ term that you multiplied by. For example, multiply both sides of the desired equation by $\cos(3x)$ and integrate over $[0,2\pi]$ to obtain $$ \begin{align} \int_{0}^{2\pi}f(x)\cos(3x)dx & = a_{0}\int_{0}^{2\pi}\cos(3x)dx \\ & +a_{1}\int_{0}^{2\pi}\cos(x)\cos(3x)dx+b_{1}\int_{0}^{2\pi}\sin(x)\cos(3x)dx+ \\ & +a_{2}\int_{0}^{2\pi}\cos(2x)\cos(3x)dx+b_{2}\int_{0}^{2\pi}\sin(2x)\cos(3x)dx+\\ & +\cdots + \\ & +a_{n}\int_{0}^{2\pi}\cos(nx)\cos(3x)dx+b_{n}\int_{0}^{2\pi}\sin(nx)\cos(3x)dx+\\ & + \cdots \end{align} $$ The only term on the right that is non-zero is $$ a_{3}\int_{0}^{2\pi}\cos(3x)\cos(3x)dx. $$ Therefore, if $f$ had such an expansion, Euler and Clairaut reasoned that it would be necessary to have, for example, $$ \int_{0}^{2\pi}f(x)\cos(3x)dx = a_{3}\int_{0}^{2\pi}\cos^{2}(3x)dx. $$ This would then lead to a unique expression for the coefficient $$ a_{3} = \frac{\int_{0}^{2\pi}f(x)\cos(3x)dx}{\int_{0}^{2\pi}\cos(3x)\cos(3x)dx}= \frac{1}{\pi}\int_{0}^{2\pi}f(x)\cos(3x)dx. $$ There are lots of issues in justifying this process, including whether or not it is valid to interchange integration and infinite summation. But this is definitely a compelling argument for believing that the coefficients would be uniquely determined. Fourier knew about this argument, and he found that this situation of "orthogonality" was the norm when dealing with solutions of his Heat Equation, whether dealing with trigonometric functions or with more general solutions coming out of Separation of Variables.

In sharp contrast to Fourier's beliefs, Mathematicians at the time believed that such coefficient equations could not always be satisfied--they believed that $f$ would have to be of some special form in order for the required conditions to hold. Fourier took a different point of view: he believed that if you used the derived coefficient formulae, you would end up with a series that would converge to $f$, for any $f$. That was controversial at the time, and most of the Mathematical powers that be at the time dismissed Fourier's conjecture as false, and they went so far as to ban his original Treatise on Heat Conduction from being published for almost 20 years, until Fourier gained prominence and forced its publication in original form.

Fourier took the approach of starting with the expressions for the coefficients that he reasoned were necessary, and then showing that the resulting series would converge. That was an interesting new twist. In other words, he took as a starting point the series $$ f \sim \frac{\int_{0}^{2\pi}f(y)dy}{\int_{0}^{2\pi}dy}+\\ +\sum_{n=0}^{\infty}\frac{\int_{0}^{2\pi}f(y)\cos(nx)dy}{\int_{0}^{2\pi}\cos(ny)\cos(ny)dy}\cos(nx)+\frac{\int_{0}^{2\pi}f(y)\sin(ny)dy}{\int_{0}^{2\pi}\sin(ny)\sin(ny)dy}\sin(nx) $$ Fourier then tried to argue that this particular series and various other series of this type would converge to $f$. He succeeded for many cases, and he came up with the Dirichlet integral for the series truncated at $n=N$. (It is now known not to have originated with Dirichlet, but was found 20 years earlier in Fourier's original work, along with a restricted convergence proof that is basically correct, and virtually the same as the proof credited to Dirichlet 20 years later. Fourier's work was not well-known during that time because it had been banned from publication! Dirichlet is believed to have had access to the unpublished work.)