Separability of Reproducing Kernel Hilbert Space of a positive definite covariance function.

449 Views Asked by At

I am reading the book Random Fields and Geometry by R. Adler and J. Taylor. In chapter 3 they introduce the concept of reproducing kernel Hilbert space (RKHS). I will summarize the definition in what follows.

Firstly let us consider a Gaussian process $f$ defined on $T$ a metric space which we assume compact in the canonical metric $d$ generated by the process, where $$ d(s, t) = \sqrt{\mathbb{E}[(f(s)-f(t))^2]} $$ for $s,t \in T$. Let $C(s,t)$ be the covariance function of $f$ and let $C$ be continuous and positive definite. Let us define $$ S = \left\{ u \colon \ T \rightarrow \mathbb{R} \colon u(\cdot) = \sum_{n=1}^n a_i C (s_i, \cdot), a_i \in \mathbb{R}, s_i \in T, n \geq 1 \right \}. $$ Define an inner product on $S$ by $$ (u,v)_H = \left( \sum_{i=1}^n a_i C(s_i, \cdot), \sum_{j=1}^m b_j C(t_j, \cdot)\right)_H. $$ Note that, since we assumed that $C$ is positive definite $(\cdot, \cdot)_H$ induces a norm. Moreover the reproducing kernel property holds, namely $$ (u, C(t, \cdot)_H = \left( \sum_{i=1}^n a_i C(s_i, \cdot), C(t,\cdot)\right)_H = \sum_{i=1}^n a_i C(s_i, t)=u(t). $$ Let us define $H(C)$ the reproducing kernel hilbert space of $f$ the closure of $S$ under the norm induced by the inner product $( \cdot, \cdot)_H$.

The the authors state that since $T$ is separable (it is a compact metric space) then $H(C)$ is separable due to the continuity of $C$, how does if follows? My first try was: let $\{\phi_n\}_{n \in \mathbb{N}}$ be the orthonormal sequence on $T$, then I would like to say that $\psi_n(\cdot) = C (\phi_n, \cdot)$ is an orthonormal sequence on $H(C)$, but here I stop. Moreover is the inner product unique?

1

There are 1 best solutions below

0
On BEST ANSWER

From the reproducing property of the RKHS it follows that the canonical feature map $\Phi:T \to H(C):t \mapsto C(\cdot ,t)$ is continuous, since if $t_n \to t$ in $T$, then $$ ||C(\cdot,t_n)-C(\cdot,t)||_H^2 = \underbrace{(C(\cdot,t_n),C(\cdot,t_n))_H}_{C(t_n,t_n)}- 2\underbrace{(C(\cdot,t_n),C(\cdot,t))_H}_{C(t_n,t)} + \underbrace{(C(\cdot,t),C(\cdot,t))_H}_{C(t,t)} = d(t_n,t)^2 \to 0\,. $$ But by definition of $H(C)$ we have $H(C) = \overline{\text{span}}\,\Phi(T)$ and since $T$ is separable and $\Phi$ is continuous, the set $\Phi(T)$ is separable and hence also $H(C)$.

One can show that actually $H(C)$ is really a space of functions, i.e. it can be made to consist of true functions $T \to \mathbb{R}$ and not only equivalence classes of sequences.