Suppose we have a weakly stationary Gaussian Process $X_t$ with zero mean and covariance function $K(t-s)$. Then the (mean square) derivative process $Y_t$ of $X_t$ has covariance function $-K''(t-s)$, provided $K$ is twice differentiable.
My question is, is there an analogue for (some) integral process of $X_t$ that preserves weak stationarity? In particular does there exist a process $Z_t$ such that $Z_t$ has mean square derivative $X_t$ and $Z_t$ is weak stationary? So if $Z_t$ has covariance function $C(t-s)$, then $K(t-s)=-C''(t-s)$? In general it seems like weak stationarity is not preserved. For example, consider
$K(t-s)=exp(-|t-s|)$
Assume without loss of generality that $t \geq s$, we have:
$$\int_{0}^{s}\int_{0}^{t} exp(-|t'-s'|) \,dt'ds'=\int_{0}^{s}\int_{0}^{s'} exp(-|t'-s'|) \,dt'ds'+\int_{0}^{s}\int_{s'}^{t} exp(-|t'-s'|) \,dt'ds'$$ $$=\int_{0}^{s}\int_{0}^{s'} exp(-(s'-t')) \,dt'ds'+\int_{0}^{s}\int_{s'}^{t} exp(-(t'-s')) \,dt'ds'$$
$$=\int_{0}^{s}(1-exp(-s'))ds'+\int_{0}^{s}(-exp(-(t-s'))+1) ds'$$ $$=2s-exp(-(t-s))+exp(-s)+exp(-t)-1$$
which is clearly not stationary. I feel like I'm missing something obvious because stationarity is preserved under differentiation yet doesn't seem to be under integration?