What is the intuition behind summability kernel and convolution?

Question

In Fourier analysis, when I look at the theorems and useful results derived using summability kernel and convolution, I get to think

"Ok, I guess it works that way. but what is the intuition behind all those complicated looking definitions?"

To a beginner like me, the three properties of summability kernel, or the definition of convolution looks just not so intuitive. I wonder "why do I even care about such things?"

I get how they are used to derive good results, but I wonder how people got to those definitions from the start at all.

Can anyone explain please?

Answer 1

Given that this topic is best treated in a full course or book, I'll just give a few pointers:

A first way of looking at Fourier is through the "harmonic analysis" lens (Wikipedia) : you have a "basis" of functions $(e_k)_k$ (in classical Fourier $e_k(x) = e^{ikx}$) and you "decompose" a generic function $f$ along your basis through some "projection" operation $(f,e_k)$ (in Fourier $\hat f(k) = (f,e_k) = \int f(x) \bar{e_k}(x)dx$). When your basis is well-chosen, your decomposition reveals some information on $f$ and, even better, you can reconstruct $f$ from the $(f,e_k)$ terms (in Fourier transform, it looks like $\hat{\hat f} \sim f$). One remarkable example of "revealed information" (or correspondence) is that the higher the regularity of $f$, the faster $\hat f$ converges to zero (Wikipedia).

A second way is to consider Fourier as a simple convolution (Wikipedia). Given that the Fourier operation $(f,e_k)$ is linear, the $n$-th partial Fourier series approximation reads $S_n[f](x) = \sum_{-n}^n \hat f(k)e^{ikx} = \sum e^{ikx}\int f(t)e^{-ikt}dt = \int f(t) \sum e^{ik(x-t)}dt = f \star D_n (x)$ where $D_n$ (= the term in the sum) is the Dirichlet kernel.

So linearly decomposing over a base is equivalent to convoluting with some function. Both angles are interesting and reveal something about harmonic analysis.

The beauty of the Dirichlet kernel (or the Fejer kernel) is that it has fixed integral of $1$ and "converges" towards the Dirac delta function, which is just another way of saying that $f \star D_n \to f$ point-wise. Convolution as a way to smooth functions or to approach of a generic function with smooth functions is a very general and important idea that reaches far beyond harmonic analysis. You can for instance use it to prove the Weierstrass theorem by building a polynomial approximation for the Dirac delta (see for instance this link).

How these definitions appeared in history is probably not the most important part; rather try to see how they borrow concepts from different parts of maths and enlighten each other. Think of harmonic decomposition as inspired by geometric projections and linear algebra (how you can decompose a vector $x$ on a basis $e_i$ with coordinates $x_i = (x,e_i)$). Convolution is quite a natural idea in many domains (think of how you smooth an image by average each pixel with its neighbors)...

Answer 2

Around the mid 1700's people were looking at describing the motions of a section of vibrating string between points $x=0$ and $x=2\pi$, and they found solutions $d(t,x)=\sin(kt)\sin(nx)$, etc., where $d$ is the displacement of the string for $0 \le x \le \pi$ and $t$ is time. In order to describe an arbitrary displacement function $f(x)$ at time $t=0$, they turned to studying the following equations: $$ f(x) = a_{0}+a_{1}\cos x+b_1 \sin x + a_2\cos 2x + b_2\sin 2x + \cdots, \;\;\; 0 \le x \le 2\pi. $$ If they could figure out how to write $f$ in this way, then they would know the displacements of the string at a later time by multiplying each term by a corresponding time function. The problem was to solve the above equation. Remarkably, Euler and Clairaut discovered an integral condition for $a_{j}$, $b_{j}$ could be obtained by multiplying by one of the $\sin$ or $\cos$ terms and integrating the product over $[0,2\pi]$. All of the terms would drop out (i.e., would give zero) except for the one involving that particular $\cos$ or $\sin$ term; and that allowed them to isolate the coefficient of that term. For example, $$ \int_{0}^{2\pi}f(x)\cos(nx)dx = a_{n}\int_{0}^{2\pi}\cos(nx)\cos(nx)dx = \pi a_{n} \\ a_{n} = \frac{1}{\pi}\int_{0}^{2\pi}f(x)\cos(nx)dx. $$ Fast forward about 40 or 50 years ...

Fourier had come up with his heat equation to describe the flow of heat in space or matter. In order to solve his equation, he proposed writing solutions as separated solutions $T(t)X(x)$ (obviously related to what was already known) and to try to write the final solution as constants times these. Fourier started with the classical trigonometric series, took what he knew would have to be the coefficients, and he studied the convergence of the truncated series that now bears his name: $$ \begin{align} S_{N}^{f}(x) & = \frac{1}{2\pi}\int_{0}^{2\pi}f(x)dx \\ & + \sum_{n=1}^{N}\frac{1}{\pi}\int_{0}^{2\pi}f(x')\cos(nx')dx'\cos(nx) \\ & +\sum_{n=1}^{N}\frac{1}{\pi}\int_{0}^{2\pi}f(x')\sin(nx')dx'\sin(nx). \end{align} $$ Fourier actually discovered the sum of this truncated series to be $$ S_{N}^{f}(x) = \frac{1}{2\pi}\int_{0}^{2\pi}\frac{\sin(N+\frac{1}{2})(x-x')}{\sin\frac{1}{2}(x-x')}f(x')dx'. $$ Dirichlet used the same formula to study the series about 20-25 years later, and he was credited with the first proof of pointwise convergence under reasonable assumptions on $f$. Historians have corrected this bit of history: Fourier discovered this formula in his original work, but it was banned from publication for over 20 years, and Fourier had come up with almost the same proof; there is reason to believe that Dirichlet may have had access to Fourier's unpublished work. Both observed that the integral of the kernel was 1 (obvious from the original sum and the coefficient relations) and that the integral away from $x'=x$ vanishes in the limit as $N\rightarrow\infty$. So it's natural to think that the limit for large $N$ would give the value of $f(x)$ if $f$ is smooth near $x$.

Much later, Abel, a brilliant Mathematician who is credited with much of the foundational work for group theory, studied summability methods to deal with conditionally convergent and even divergent series. One method Abel used for studying the convergence of a series $\{ s_{n} \}_{n=1}^{\infty}$ was to consider the sequence of running averages of the series instead: $$ S_{N} = \frac{1}{N}\left[s_{0} + s_{1} + \cdots + s_{N-1}\right]. $$ So naturally this was eventually applied to the Fourier series, which had become an important problem in Mathematics by that time, where conditions of general convergence were elusive. When you work this out for the Fourier series, you get the Fejer kernel (named after the Mathematician who applied Abel's method to the Fourier Series.) Fejer's integral is $$ \frac{1}{N}\left[S_{0}^{f}+S_{1}^{f}+\cdots+S_{N-1}^{f}\right]= \frac{1}{2\pi N}\int_{0}^{\pi}\left(\frac{\sin\frac{N}{2}(x'-x)}{\sin\frac{1}{2}(x'-x)}\right)^{2}f(x')dx'. $$ This kernel also has total integral one for obvious reasons, but this kernel function is always positive, which makes convergence much easier to study. So, as one might reasonably expect, Abel summability improves the convergence of the original Fourier series. In fact, the averaged series converges if $f$ is mereley continuous at a point or has left and right hand limits; that's not the case for the general Fourier series.