If all features of a kernel $k$ are contained in a RKHS $H$, does then $H(k)\subset H$?

36 Views Asked by At

Let $k_1$ and $k_2$ be symmetric positive definite kernels on a set $X$, denote the corresponding RKHS's by $H_1$ and $H_2$, respectively, and also denote the canonical feature maps by $\Phi_1 : X\to H_1$ and $\Phi_2 : X\to H_2$, respectively.

Assume that $\Phi_1(x)\in H_2$ for all $x\in X$. Does it then follow that $H_1\subset H_2$?

My kernels are actually Gaussian, so we may assume that $X$ is a subset of $\mathbb R^d$, for example $\mathbb R^d$ itself or the closed or open unit ball.

I tried a lot, but either I am too dumb to prove this, or it is just false. At least I know that the condition gives me $$ \sum_{i,j=1}^n\alpha_i\alpha_j k_1(x,y_i)k_1(x,y_j)\,\le\,c^2\sum_{i,j=1}^n\alpha_i\alpha_j k_2(y_iy_j) $$ for some $c>0$ and all $\alpha_i\in\mathbb R$, all $y_i\in X$ and all $x\in X$. However, what I want is $$ \sum_{i,j=1}^n\alpha_i\alpha_j k_1(y_i,y_j)\,\le\,c^2\sum_{i,j=1}^n\alpha_i\alpha_j k_2(y_iy_j), $$ because this is equivalent to $H_1\subset H_2$.

In the case of Gaussian kernels, I tried to integrate the first one w.r.t. some probability measure over $X=\mathbb R^d$, but didn't succeed.