I'm having trouble with kernels:https://en.wikipedia.org/wiki/Positive-definite_kernel, and have two small questions
1.) Is it correct to say any positive definite matrix can make a kernel.
eg.this is the definition of a positive definite kernel,
$$ \sum_{i=1}^{n} \sum_{j=1}^{n} c_{i} c_{j} K\left(\mathbf{x_{i}}, \mathbf{x_{j}}\right) \geq 0 $$
can I set $$K\left(\mathbf{x_{i}}, \mathbf{x_{j}}\right) $$to $$\mathbf{x_{i}}^{T}\mathbf{Ax_{j}}$$, where A is positive definite. And the above hold?
- If this is true can I write $$ \sum_{i=1}^{n} \sum_{j=1}^{n} c_{i} c_{j} \mathbf{x_{i}}^{T}\mathbf{Ax_{j}} \geq 0 $$ as a new kernel.
$$ \sum_{i=1}^{n} \sum_{j=1}^{n}K_{n}\left(\mathbf{x_{i}}, \mathbf{x_{j}}\right) \geq 0 $$
where $$ K_{n} = c_{i} c_{j} \mathbf{x_{i}}^{T}\mathbf{Ax_{j}} \geq 0 ?$$
Yes, for any positive definite matrix $A$, the kernel as $K(x_i,x_j) = x_i^TAx_j$ is positive definite. The proof is quite simple. For any $c_i$, $i=1,2,\dots,n$, we have $$\sum_i\sum_j c_ic_jK(x_i,x_j) = \sum_i\sum_j c_ic_jx_i^TAx_j = s^TAs\ge 0,$$ where $s = \sum_i c_ix_i$.
Your question doesn't really make sense, because the $c_i$ variables in the definition are not fixed, predefined variables. The definition requires that a certain inequality holds for any vector $c = (c_1,\dots,c_n)$. Now if the question is: "For a given vector $c = (c_1,\dots,c_n)$, is the kernel $K(x_i,x_j) = c_ic_jx_i^TAx_j$ positive definite?" then we need to check whether the inequality $\sum_i\sum_j c'_ic'_jK(x_i,x_j) \ge 0$ holds for any vector $c'=(c'_1,\dots,c'_n)$ (which has nothing to do with vector $c$). The answer is also yes. Same proof as above, just use $s = \sum_i c_i'c_ix_i$.