How to show these covariance functions form a kernel? (I.e. a covariance matrix for any finite set of points)

89 Views Asked by At

In machine learning (specifically, Gaussian processes), a "kernel" is a two argument function such that for any set of $N$ "input points," (any $N$, any points), the $N \times N$ matrix of pairwise kernel function values for every pair of points forms a valid covariance matrix, i.e. symmetric positive semi-definite.

I have a slightly special case in that for each input (real number) $t$, there are two random variables $a_t$ and $b_t$ so for $N$ inputs my matrix is $2n \times 2n$.

My kernel came from the idea of having a "local linear polynomial fit" at $t$ where the polynomial is $f_t(x)a_t(x-t) + b_t$. The coefficients $a_t$ and $b_t$ are random variables where $a_t$ is modeled as an infinite mixture (integral) of independent infinitesimal random variables each indexed by a real number $t_0$ that have an exponential decay factor of contribution to $a_t$ as $|t-t_0|$ increases. $b_t$ is a bit more complex, it is also an integral of (new) independent infinitesimal random variables that have exponential decay, but also take into account the "linear" effect by integrating each infinitesimal variable that contributed to the $a_t$ and adding these quantities to the contribution to $b_t$. My kernel function $K$ is thus defined as:

$$K(a_{t_1},a_{t_2}) = \int_{-\infty}^\infty e^{-\alpha(|t_0 = t_1| + |t_0-t_2|)} dt_0$$

$$K(b_{t_1},b_{t_2}) = \int_{-\infty}^\infty e^{-\beta(|t_0 = t_1| + |t_0-t_2|)} + \int_{t_0}^{t_1} \int_{t_0}^{t_2} e^{-\alpha (|y_1 - t_0| + |y_2 - t_0|) -\beta (|y_1 - t_1| + |y_2 - t_2|)} dy_1 dy_2 dt_0$$

$$K(a_{t_1},b_{t_2}) = \int_{-\infty}^\infty e^{-\alpha |t_1 - t_0|}\int_{t_0}^{t_2} e^{-\alpha |y - t_0| -\beta |y - t_2|}dy dt_0$$

Here $\alpha,\beta$ are arbitrary positive real constants that encode the decay rates.

These integral formulations are much shorter than the explicit solutions, and in any event, I think reasoning in terms of the integrals (without evaluating them) may be sufficient to prove $K$ is a kernel. In all of my experiments with a collection of input points $t_i$ (chosen randomly), I have gotten a valid covariance matrix. So I strongly suspect $K$ is indeed a valid kernel. But I don't know how to prove it?