What are the connections (if any) between "kernel" in "kernel density estimation," "kernel of a matrix," and "kernel method"?

Question

I'm getting my understanding of kernel density estimation from pages 6-7 of this PDF.

If there are conceptual relationships between the "kernels" in each of these topics, I'd like to understand them.

Answer 1

Kernel density estimation and Kernel methods(referring to interpolation or regression by using kernels) employ the same use of kernel. The kernel of a matrix is unrelated.

In the first two problems the kernel acts a basis function centered at a data points $k(x, x_i)$. The idea is that if we take linear combinations of these basis functions we can describe other functions $f(x) = \sum_i w_ik(x,x_i)$

In kernel density estimation we approximate a pdf as the average of a bunch of these basis functions. Typically people use radial basis functions $k(x,x_i) = \exp(-||x-x_i||^2/h)$

To get a good fit we need to tune the length scale parameters $h$. For general curve fitting with basis functions we do basically the same process but typically do a weighted average.