Showing that all Kernel functions are symmetric

Question

I want to show that any Kernel function is symmetric, i.e. $K(x_i, x_j) = K(x_j, x_i)$. I am not sure how I should begin, any tips will help, thanks.

Answer 1

As an aside, usually kernels in statistics or signal processing are symmetric by definition, but since you tag with machine learning, I suppose you are interested in that case, specifically the kernel trick.

Suppose $x,y\in\mathcal{D}$ are data points in the input space. We also have a feature map $\Phi:\mathcal{D}\rightarrow \mathscr{P}$, where $\mathscr{P}$ is an inner product space. Then the kernel function $K:\mathcal{D}\times\mathcal{D}\rightarrow\mathbb{R}$ is defined by: $$ K(x,y) = \langle\Phi(x),\Phi(y)\rangle_{\mathscr{P}} $$ where $\langle\cdot\rangle_{\mathscr{P}}$ is the inner product of $\mathscr{P}$.

The important fact is that $\langle\cdot\rangle_{\mathscr{P}}$ is symmetric. Thus: $$ K(x,y) = \langle\Phi(x),\Phi(y)\rangle_{\mathscr{P}} = \langle\Phi(y),\Phi(x)\rangle_{\mathscr{P}} = K(y,x) $$ so the kernel is symmetric.