Given a stochastic process $\{X_k\}_{k \geq 1}$, we say it is weakly stationary if $\mathbb{E}X_k$ is a constant, and $\mathrm{Cov}(X_k, X_l)$ only depends on $|k - l|$.
My question is, for a weakly stationary process, given any $h \geq 1$, does the conditional covariance $\mathrm{Cov}(X_k, X_{k + h}|X_{k + 1}, \cdots, X_{k + h - 1})$ depend on $k$?
I believe the answer is no if the process is strongly stationary, i.e. the joint probability distribution of a selected set of random variables doesn't change with an index shift. But I am not so sure for a weakly stationary process.
The motivation is that, if the answer to my question is no, then I could define conditional covariance of $X_{k}$ and $X_{k + h}$ from a weakly stationary process as a function of $h$ only.
If a stochastic process $\{X_k\}k≥1$ is weakly stationary, then the conditional covariance $Cov(X_k,X_{k+h}|X_{k+1},⋯,X_{k+h−1})$ does not depend on $k$.
using the definition of weak stationarity and the properties of conditional covariance.
First, we can define weak stationarity as follows: A process $\{X_k\}k≥1$ is weakly stationary if the mean E[Xk] and the autocovariance function γ(h) = $Cov(Xk,Xk+h)$ are constant for all k and h
Next, we can use the definition of conditional covariance to express $Cov(Xk, Xk+h|Xk+1,⋯, Xk+h−1)$ in terms of the unconditional mean and covariance of the process:
$Cov(Xk,Xk+h|Xk+1,⋯,Xk+h−1) =$ $E[(Xk-E[Xk])(Xk+h-E[Xk+h])|Xk+1,⋯,Xk+h−1]- $ $E[Xk|Xk+1,⋯,Xk+h−1]E[Xk+h|Xk+1,⋯,Xk+h−1]$
Since the mean and autocovariance of the weakly stationary process are constant, the above expression doesn't depend on k. We know that the conditional mean of Xk given the past values is equal to the unconditional mean since the past values are assumed to be independent of the mean of the process.
And also, the conditional autocovariance is still equal to the unconditional autocovariance function, since the past values are independent of the autocovariance function. Therefore, the expression for the conditional covariance simplifies to: $Cov(Xk,Xk+h|Xk+1,⋯,Xk+h−1)$ = γ(h) - E[Xk]E[Xk+h]$
As you can see, this expression does not depend on k, which means that the conditional covariance is a function of h only, as stated before.