I'm reading the book "High Dimensional Statistics" by Martin Wainwright just for fun (also as preparation of my PhD in computer science/Machine Learning). In particular, I'm currently reading chapter 12 on Reproducing Kernel Hilbert Spaces (RKHS's).
In section 12.2.3, he shows how to construct an RKHS from a Kernel, but I have a fundamental analysis question on the construction.
He considers the space $\mathbb{H}$ of functions of the form $$ f(.) = \sum_{j=1}^{n}\alpha_j K(.,x_j)$$
for some integer $n\geq 1$ and a set of points $\{x_j\}_{j=1}^{n} \subset X$ and weight vector $\alpha \in \mathbb{R}^n$. $K$ here refers to any positive semi-definite kernel $$ K: X \times X \to \mathbb{R}$$
For any $f, g \in \mathbb{H}$ represented as $f(.) = \sum_{j=1}^{n}\alpha_j K(.,x_j)$ and $g(.) = \sum_{k=1}^{n}\beta_k K(.,x_k)$ respectively, the author defines the inner product as
$$\langle f, g \rangle_{\mathbb{H}} = \sum_{j=1}^{n}\sum_{k=1}^{n}\alpha_j \beta_k K(x_j,x_k)$$
And it is relatively trivial to show that $\langle.,.\rangle_{\mathbb{H}}$ defines a valid innner product. Moreover, this inner product satisfies the kernel reproducing property since
$$\langle f, K(.,x)\rangle_{\mathbb{H}} = \sum_{j=1}^{n}\alpha_jK(x_j,x) = f(x)$$
All of this makes sense. The part that I don't follow quite exactly is how to extend this space $\mathbb{H}$ with this inner product to a complete space (i.e., have all Cauchy sequences converge).
In the proof of this metric space completion, the author states that "if $(f_n)_{n=1}^{\infty}$ is a Cauchy sequence in $\mathbb{H}$, then for each $x \in X$, the sequence $(f_n(x))_{n=1}^{\infty}$ is Cauchy in $\mathbb{R}$."
This is the part that I've been having trouble understanding so far. Don't we need to assume that the sequence $(f_n)_{n=1}^{\infty}$ is pointwise bounded and equicontinuous for this to be true? If we were to make that assumption, then we can use Arzela-Ascoli to get a uninformly convergent subsequence, which also implies pointwise convergence.
Can someone help me understand why that statement is true without further assumptions on the functions $f_n$? Thanks!
In that context, the convergence (hence Cauchy-ness) of $(f_n)_{n=1}^{\infty}$ is not measured by the supremum norm $\|f\|_{\sup} = \sup_{x\in X}|f(x)|$ as is the case in Arzelà–Ascoli theorem.
Rather, it is measured via the norm
$$ \| f\|_{\mathbb{H}} = \sqrt{\langle f, f\rangle_{\mathbb{H}}} $$
induced by the inner product on $\mathbb{H}$. Hence, that $(f_n)_{n=1}^{\infty}$ is Cauchy in $\mathbb{H}$ really means that
$$ \| f_n - f_m \|_{\mathbb{H}} \to 0 \quad\text{as}\quad m, n \to \infty. $$
Now, this implies the Cauchy-ness of $(f(x))_{n=1}^{\infty}$ for each $x \in X$ because we have
$$ |f_n(x) - f_m(x)| = |\langle f_n - f_m, K(\cdot, x)\rangle_{\mathbb{H}}| \leq \|f_n - f_m\|_{\mathbb{H}} \|K(\cdot, x)\|_{\mathbb{H}} \to 0 $$
as $m, n \to \infty$ by Cauchy–Schwarz inequality.