The formal definition I've been given relating to weakly stationary processes follows:
Definition. A stochastic process $X = (X_t, t \in T)$ is weakly stationary if
- $\forall t \in T, E[X_t^2] < \infty \, ;$
- $\forall t \in T, E[X_t] = \mu $ (independent of t) $;$
- $\forall t,s \in T, Cov(X_s,X_t) $ depends only on the lag $|t-s|.$
Intuitively, what I understand from point $2.$ is that the mean function of $X$ must be finite and equal for every instant $t \in T$. On the other hand, from point $1.$ I understand that $E[X_t^2]$ must be finite for all $t \in T$, but not necessarily equal for all $t \in T$. For example, we could have $E[X_1^2] = 1 < \infty$ and $E[X_t^2] = 0 < \infty, \forall t \in T \setminus \{1\}$ and point $1.$ would still be satisfied. Is my intuition correct on these two topics?
I am getting doubts about my intuiton in this definition because, in my class, the autocorrelation function was defined as follows:
Definition. The autocorrelation function of a stochastic process $X = (X_t, t \in T)$ is given by
$$ \rho(h) = \frac{Cov(X_{t+h},X_t)}{\sqrt{V(X_{t+h}})\sqrt{V(X_t)}} = \frac{\gamma(h)}{\gamma(0)},$$
where $\gamma$ represents the autocovariance function. It was also stated in class the the last equality is only valid if the stochastic process in question is weakly stationary.
Now, if my intuition on point $1.$ from the first definition is correct, this starts to be quite confusing. Being more detailed, the second equality on the definition of the autocorrelation function makes no sense if I allow my intuition on point $1.$ to be correct, since if I do so, the variance might not be equal for two different instants $t \in T$.
No, the second moment must be constant, as implied by point (3). Notice that $$Var(X_t)= Cov(X_t,X_t) = Cov(X_s, X_s)$$ for any $t,s\in T$. Hence $E(X_t^2) = Var(X_t) + E(X_t)^2$ is constant.