I've run up against a wall in reconciling two different definitions of the Ornstein–Uhlenbeck process, and would appreciate some help.
On the one hand, as discussed here, we can define an Ornstein–Uhlenbeck process as a Gaussian process with a kernel function of the form $k(x, y) \propto \exp(-\left\lVert x - y\right\rVert / \theta)$, which is clearly stationary.
On the other hand, we have the definition of the Ornstein–Uhlenbeck process as the solution to the stochastic differential equation $du(t) = \theta(\mu - u(t))+\sigma \, dW(t)$, which is given by
$$u(t)= u(0) \exp(-\theta t)+\mu(1-\exp(-\theta t) )+\sigma \exp(-\theta t) \int_0^t \exp(\theta\tau) \, dW(\tau).$$
Using the Itô isometry, as shown here and here on SE, this process can be shown to have the covariance $$k(x, y) = \frac{\sigma^2}{2\theta} \exp(-\theta (x+y)) (1 - \exp(2 \theta (x - y))).$$
Not only is this not equal to the previous definition, this is not even stationary. What gives? Are these two distinct definitions of what "Ornstein–Uhlenbeck process" means, or can these two processes be shown to be equal, possibly modulo certain assumptions?
As a bonus question, just because I've never seen anyone prove it - how do you prove that the solution to the above SDE is a Gaussian process (assuming it is)?