Linear kernel is strictly positive definite, but corresponding RKHS in not dense in C(R)?

338 Views Asked by At

There must be something very basic that I am missing.

If I understood correctly from wikipedia, positive-definiteness of kernel $K(\cdot,\cdot)$ is sufficient for universality of this kernel. Then, according to same wiki page, corresponding Reproducing Kernel Hilbert Space (RKHS) $\mathcal H_K$ must be dense in $\mathcal C(\mathbb R)$. More specifically, for each kernel $K$ we can build corresponding RKHS as following:

$$\mathcal H_K = \text{span}\{K(x, .) : x \in \mathbb R\}$$ with scalar product given by

$$ f(x) = \sum_i^n \alpha_i K(x_i, x)$$ $$ g(x) = \sum_j^m \beta_j K(x_j, x)$$

$$ f \cdot g = \sum_i^n \sum_j^m \beta_j \alpha_i K(x_j, x) \cdot K(x_i, x) = \sum_i^n \sum_j^m \beta_j \alpha_i K(x_j, x_i) $$

However, even though $K(x, y) = x^T y$ is positive definite, it seem to consist only of linear functions, so its closure is far from being $\mathcal C(\mathbb R)$.

1

There are 1 best solutions below

1
On BEST ANSWER

Now I have figured out, thank you, @JeanMarie.

Several mistakes I made:

$$ \mathcal H = \overline{\text{span}\{K(x, \cdot), x \in X\}}$$, so $\mathcal H$ has infinite sums.

Second mistake and answer to my question [1, page 139]:

If kernel is shift-invariant $h(x-y)=K(x, y)$ and we represent it in Fourier domain as $$h(t) = \int e^{-i\langle t, \omega \rangle} \mu(d\omega)$$ then if $\text{supp}\{\mu\}$ is entire space, then $K$ is universal, so $H_K$ is dense in the space of all bounded continuous functions.