Understanding convolution

Question

Take:

$$ (u*v)(k) = \sum_{i=-\infty}^\infty u(i)v(k-i). $$

The $k$ is there, it's because you want to define $$ \ldots\ldots, (u*v)(-3), (u*v)(-2), (u*v)(-1), (u*v)(0), (u*v)(1), (u*v)(2), (u*v)(3), \ldots\ldots >$$ etc. The number in the parentheses is $k$. Thus, for example, when $k=4$, we have \begin{align} (u*v)(4) = \sum_{i=-\infty}^\infty u(i)v(4-i) \end{align} $$ = \cdots\cdots+u(-3)v(7)+u(-2)v(6)+u(-1)v(5)+u(0)v(4)+u(1)v(3)+u(2)v(2) >$$ $$ \phantom{={}} {}+u(3)v(1)+ u(4)v(0) + u(5) v(-1) + u(6)v(-2)+u(7)v(-3)+u(8)v(-4)+\cdots>\cdots. >$$

Could you give examples of what the 'k' value could stand for? Can it only be time offset?

Could you give an intuitive explaination why you need to multiply u(-3) with u(7)! And not u(-3) with v(-3) for example? With a diagram for example.

Answer 1

I always found it more insightful to look at convolution in a way that's not quite obvious from the usual definitions. However, at least in the discrete case as you her brought forth, it's a completely equivalent, if somewhat weird-looking definition: $$ (u_1 \ast u_2 \ast \ldots \ast u_n)(k) = \sum_{(i_1,i_2\ldots,i_n)\in \mathbb{Z}^n\colon \sum_m i_m = k} \prod_{m=1}^n u_m(i_m) $$ or, for just two functions, $$ (u \ast v)(k) = \sum_{(i,j)\in \mathbb{Z}^2 \colon i+j=k}\!\! u(i)\cdot v(j) $$ So you're considering the product of results from (all) the given functions, summed over all ways to evaluate them such that the arguments sum up to $k$.

Not sure if that answers your question, but...

Answer 2

This may be more than you want, but if these sequences are "on the spectral side", that is, are Fourier coefficients of functions, say $U(x)=\sum_m u_m\,e^{imx}$ and $V(x)=\sum_n v_n\,e^{inx}$, then the usual pointwise multiplication of functions has the effect on Fourier coefficients of convolution. That is, the function $UV(x)=U(x)\cdot V(x)$ has $k$th Fourier coefficient $\sum_j u_{k-j}\cdot v_j$. This follows by a direct computation.

Although this mechanism can obviously be abstracted and formalized, forgetting about Fourier series, etc., I think this structure is the real reason we care. There are other discretized variants, too, but this is the simplest, most natural, and historically first, I think.