What weak stationarity tells use about the variance.

Question

As a start, I'm sorry if I utilize some of these terms terribly, as I'm very unfamiliar with this field.

If a series y_t,y_t-1, ... is weakly stationary. What would this tell use about limit of var(y_t) as t approaches infinity? I found something that says the limit of E[y_t] becomes a constant, but all i can find is that var(y_t) "converges", now I'm not sure if that is supposed to mean it becomes 0 or becomes a fixed value (which may be 0).

Answer 1

$\DeclareMathOperator{\var}{var}$ $\DeclareMathOperator{\E}{E}$ That a stochastic process is weakly stationary means that $$ \E[ y_t ] = m(t) = m $$ and $$ \E[ (y_t - m(t)) (y_t' - m(t')) ] = c(t, t') = c(\Delta t), $$ where $m$ is a constant and $c$ is only a function of the time difference $\Delta t = t - t'$; i.e. both the expectation and the autocovariance are invariant under time shifts. A special case of the autocovariance is the variance $$ \E[ (y_t - m(t))^2 ] = c(0) = v, $$ where $v$ is a constant. The expectation $\E$ here is taken across the realizations of the stochastic process.

I assume that by $\var(y_t)$ you mean the variance across time, $$ \var(y_t) = \frac1t \sum_{t'=1}^t \left ( y_{t'} - \frac1t \sum_{t''=1}^t y_{t''} \right ) ^ 2, $$ and then you are interested in $\lim_{t \to \infty} \var(y_t)$?

As pointed out by Did in the comments, the notation $\var$ for the variance across time is misleading, since it should be referring to what I denoted by $v$ (or $v(t)$ for a non-stationary process) above. It is also known as the empirical or sample variance, and sometimes denoted by $\hat \sigma ^2 _t$. For the sake of simplicity, I will keep using the OP's notation in this answer.

The problem is that $\var(y_t)$ is a function of a particular realization of the process, and therefore it does not have a fixed value but is a random variable with a distribution in itself. There are several definitions for convergence of random variables. I think the one that applies here is "almost sure convergence", meaning here that there is a value $l$ such that $$ \mathrm{Pr} \left ( \lim_{t \to \infty} \var(y_t) = l \right ) = 1, $$ i.e. almost every realization of $\var(y_t)$ converges to $l$.

If $\var(y_t)$ converges in this sense, the limit will be equal to the constant point-wise variance of the process, $l = v$. Whether it converges will depend on the autocovariance function $c(\Delta t)$ of the process, in particular on whether and how fast it decreases to $0$ for increasing $\Delta t$ (whether there are long-range correlations).