In the construction of Reproducing Kernel Hilbert spaces via the Moore–Aronszajn theorem one uses the completion of the linear span of $\{K_x |\ x\in X\}$, where $K_x(y)=K(x,y)$ and $K$ is some continuous, positive semi-definite, symmetric kernel on a set X. The completion is taken w.r.t. the inner product defined by $<K_x,K_y>:=K(x,y)$. My Question is rather general:
Why can one characterise this completion as follows? $$ f=\sum_{i=1}^{\infty} a_iK_{x_i} \qquad (1)$$ for some $x_i\in X$ and $$ \sum_{i=1}^{\infty} a_i^2 \ K(x_i,x_i)< \infty \qquad (2)$$
The (2) assumption is due to the convergence of the series w.r.t. the inner product on the RKHS, but why can an arbitrary element of the completion $f$ be written as an infinite series? (in my knowledge completions are defined by going over to a space of series or by taking limits, not necessary series)
Dino Sejdinovic and Arthur Gretton have some great notes on this, see: http://www.stats.ox.ac.uk/~sejdinov/teaching/atml14/Theory_2014.pdf.
Basically you define a pre-RKHS which is the set of functions of the form $$ f=\sum_{i=1}^{n} a_iK_{x_i}$$ for some $x_i\in X$. And then your RKHS the completion of this space.
While every function of the form $f=\sum_{i=1}^{\infty} a_iK_{x_i}$ will then be in this space, it doesn't characterize it as there could be other functions in the space. So its not true that every element can be written in this form.