I am trying to understand the various definitions of stationarity of the OU process and I can't reconcile a step below.
Let me summarize what I know and did. So the OU process is defined by the SDE $$ dX_t=\theta(\mu-X_t)+\sigma dW_t,\tag{1} $$ where the parameters are time-independent constants. If we start the process from a known initial condition $X_0=x_0$ at $t=0$ we find $$ X_t=\mu+(x_0-\mu)e^{-\theta t}+\sigma\int_0^t e^{\theta(u-t)}dW_u,\tag{2} $$ which is a Markovian, Gaussian process with time-dependent mean $\mathbb{E}X_t$ and autocovariance $K_{XX}(s,t)=\mathbb{E}((X_s-\mathbb{E}X_s)(X_t-\mathbb{E}X_t))$ that clearly does not only depend on the time difference $|t-s|$ (see the Wikipedia page linked above). If the mean was time-independent and the autocovariance depended only on the time difference then this solution would have weak stationarity. However, if $\theta>0$, this solution admits a steady-state, time-independent PDF as $t\to\infty$ (sometimes called the stationary distribution) that reads $$ p_\infty(x)=\frac{1}{\sqrt{2\pi v}}e^{-(x-\mu)^2/(2v)},\quad v=\frac{\sigma^2}{2\theta}.\tag{3} $$ This can be derived directly from the steady-state Fokker-Planck equation or from the $t\to\infty$ limit of the conditional PDF (or transition density) $p(x,t|x_0,0)$. Now, if we start the process $X_t$ with a random $x_0$, drawn from $p_\infty(x_0)$, the resulting unconditional PDF will be time-independent, stationary; it will actually be $p_\infty$ itself $$ p_\infty(x)=\int dx_0\,p(x,t|x_0,0)p_\infty(x_0).\tag{4} $$ Now, I am interested in the actual process underlying this stationary distribution. If you simply take (2) and integrate over $x_0$ with the PDF $p_\infty(x_0)$ you get $$ X_t=\mu+\sigma\int_0^t e^{\theta(u-t)}dW_u,\tag{5} $$ whose mean is now correctly the constant $\mu$ but its variance is not the constant $v$ from (3). The variance and autocovariance remained the same as before. This of course means we can't just simply integrate $x_0$ out from (2).
It is suggested that one should get rid of the initial condition by pushing it back to the remote past $$ X_t=\mu+(x_0-\mu)e^{-\theta (t-t_0)}+\sigma\int_{t_0}^t e^{\theta(u-t)}dW_u,\quad X_{t_0}=x_0,\tag{6} $$ and in the limit $t_0\to-\infty$ we formally get $$ X_t=\mu+\sigma\int_{-\infty}^t e^{\theta(u-t)}dW_u.\tag{7} $$ This $X_t$ has the correct mean and variance/autocovariance, and should be the weak stationary process I was looking for. However, this all seems a bit formal as I don't know how to define the Brownian motion (BM) for $t<0$, in particular for $t\to-\infty$.
Questions:
- Can we get to a stationary process in (5) by integrating out $x_0$, like we did with the PDF?
- Or can we make (7) rigorous somehow, without having to rely on BM with negative times?