I am wondering how to explicitly write $K_x(\cdot)$ in RKHS when using kernel trick.
($K_x(\cdot) = K(x, \cdot)$)
This is the Moore–Aronszajn theorem.
Suppose K is a symmetric, positive definite kernel on a set X. Then there is a unique Hilbert space of functions on X for which K is a reproducing kernel.
Then Let a set X (a set of data i.e. set of 1 dimensional vector == scalar). And there exists a kernel $K: \mathcal X \times \mathcal X \rightarrow \mathbb R$ . And there exists a corresponding RKHS (space of functions : $\mathcal X \rightarrow \mathbb R)$. Then, $<K(x, \cdot ), K(y, \cdot)>_{RKHS} = K(x, y) \ \forall x, y \in \mathcal X$ bacause of the reproducing property of kernel function.
Then we can think $K(x, \cdot)$ is a FUNCTION.
I don't care about the explicit form of $<\cdot, \cdot>_{RKHS}$ but there EXISTS a corresponding RKHS with a appropriate inner product that satisfies the reproducing property because of the Moore–Aronszajn theorem.
And this is kernel trick of kernel SVM (or kernel linear regression) :
$<\phi(x), \phi(y)>_{\mathbb R^D} = K(x, y)$
And if we take RBF kernel, the RBF kernel projects scalar ($\in \mathcal X$) into an infinite dimensional vector space.
And we can figure out the explicit form of $\phi(x)$ by taylor expansion.
please see this :
https://stats.stackexchange.com/questions/69759/feature-map-for-the-gaussian-kernel
Then I am wondering whether or not $K_x(\cdot)$ and $\phi(x)$ are equivalent.
$K_x(\cdot)$ is function, so it can be interpreted as a infinite dimensional vector. (we can think the index set is infinite -> $C[0, 1]$ = x[0] ~ x[inf].. as a list of all its values over its domain).
And $\phi(x)$ is also a infinite dimensional vector
And what is the diffence of $K_x(\cdot)$ and $\phi(x)$ ?
If they are equivalent, Then how can I get the explicit form of $K_x(\cdot)$ ?
I am studying about Neural Tangent Kernel and Gaussian processes, RKHS.
But I am not good at functional analysis, I do not study functional ananlysis much.
And I am stuck at here.. I am confused. Please help me.
Thank you for reading my question. Have a good day.