Proving that convergence in RKHS implies pointwise convergence without using reproducing property

161 Views Asked by At

Let $(\mathcal{H}, \mathcal{K})$ be a reproducing kernel Hilbert space and denote $\mathcal{K}_x := \mathcal{K}(x, \cdot)$.
Is there a simple way to prove $f_n \to_\mathcal{H} f$ (shorthand for $\|f_n - f\|_\mathcal{H} \to 0$) implies $f_n \to f$ pointwise without having to invoke the reproducing property $\langle g, \mathcal{K}_x \rangle_\mathcal{H} = g(x)$?

I'm likely misunderstanding something, but it appears to me that this result is needed to prove the reproducing property itself for functions not in $\mathcal{S} := \text{span}\{\mathcal{K}_x | x \in \mathcal{X}\}$ but in its completion when constructing the RKHS $\mathcal{H}$.
It is easy to see that any function in $\mathcal{S}$ satisfies the reproducing property.
For $f \in \mathcal{H} \setminus \mathcal{S}$, by definition we can always find a sequence $f_n$ in $\mathcal{S}$ such that $f_n \to_\mathcal{H} f$. Then,

\begin{align*} \langle f, \mathcal{K}_x \rangle_\mathcal{H} &= \lim_{n \to \infty} \langle f_n, \mathcal{K}_x \rangle_\mathcal{H} \quad \text{since } \langle \cdot, \mathcal{K}_x \rangle_\mathcal{H} \text{ is continuous w.r.t } \| \cdot \|_\mathcal{H} \\ &= \lim_{n \to \infty} f_n(x) \quad \text{by the reproducing property for functions in } \mathcal{S} \\ &= f(x) \quad \text{assuming we can use the fact that } f_n \to_\mathcal{H} f \text{ implies } f_n \to f. \end{align*}

Edit:
To be clear, I'm following the proof of Theorem 12.11 from Wainwright's High-dimensional statistics, which states that

For any positive semidefinite kernel $\mathcal{K}$, there is a unique Hilbert space $\mathcal{H} \subset \mathbb{R}^\mathcal{X}$ in which the kernel satisfies the reproducing property: $$\langle f, \mathcal{K}_x \rangle_\mathcal{H} = f(x) \quad \forall f \in \mathcal{H}.$$ $\mathcal{H}$ is known as the RKHS associated with $\mathcal{K}$.

Therefore, instead of defining the RKHS to be a space of bounded evaluation functionals (this appears as a theorem in the book), the goal is to construct the RKHS by completing the span of $\{\mathcal{K}_x\}_{x \in \mathcal{X}}$ and show that it satisfies all the required properties.

1

There are 1 best solutions below

10
On

As mentioned by @daw, for a Hilbert space $\mathcal{H}$ to become a RKHS the evaluation functionals $L_{x}​(f):=f(x)$ must be continuous, which is equivalent to being bounded. Thus for every $x$ there is a $M_{x}$ with $L_{x}(f)=f(x)\leq M_{x} \| f \|_{\mathcal{H}}$.

It follows that $f_{n} \rightarrow f$ implies the pointwise convergence. Pick an arbitrary $x$ then $$ |f(x)-f_{n}(x)|=|(f-f_{n})(x)|=|L_{x}(f-f_{n})| \leq M_{x}\|f-f_{n}\| $$ and since $\|f_{n} - f\| \rightarrow 0$ we see that $f_{n}(x) \rightarrow f(x)$.