Covariance matrix of an AR(1) model?

Question

The covariance matrix of the values of the AR(1) model $X_t = \phi X_{t-1} + Z_t$ at times $t=1$ and $t=3$ is useful to find the best linear predictor of $X_2$ given $X_1$ and $X_3$.

Let $W = (X_1, X_3)^T$, what is the covariance matrix of $W$?

Thus, one asks for the variances of $X_1$ and $X_3$ and for the covariance of $X_1$ and $X_3$.

Answer 1

You probably need to assume that $X_{0} = 0$ and that $(X_{t})$ and $(Z_{t})$ are independent. I will also assume that $Z_{1},Z_{2},Z_{3}$ share a common mean $E(Z_{i}) = \mu_{Z}$.

If this is true,

Cov$(X_{1},X_{1}) = Var(X_{1}) = Var(Z_{1})$

To find $Cov(X_{1},X_{3})$ first observe that $X_{2} = \phi X_{1} + Z_{2}$, so that

$X_{3} = \phi X_{2} + Z_{3} = \phi(\phi X_{1} + Z_{2}) + Z_{3} = \phi^{2}X_{1} + \phi Z_{2} + Z_{3}$.

Hence,

$E(X_{3}) = \phi^{2}E(X_{1}) + \mu_{Z}(\phi + 1)$.

and

$E(X_{3}X_{1}) = \phi^{2}E(X_{1}^{2}) + \phi E(X_{1}Z_{2}) + E(X_{1}Z_{3}) = \phi^{2}E(X_{1}^{2}) + \mu_{Z}E(X_{1})(\phi + 1)$.

Finally, \begin{eqnarray} Cov(X_{3},X_{1}) &=& E(X_{3}X_{1}) - E(X_{3})E(X_{1}) \\ &=& \phi^{2}E(X_{1}^{2}) + \mu_{Z}E(X_{1})(\phi + 1) - \phi^{2}E(X_{1})^{2} + \mu_{Z}E(X_{1})(\phi + 1) \\ &=& \phi^{2}E(X_{1}^{2}) - \phi^{2}E(X_{1})^{2} \\ &=& \phi^{2}Var(X_{1}) \\ &=& \phi^{2}Var(Z_{1}) \end{eqnarray}

Answer 2

Assume that $(Z_t)$ is i.i.d. and centered and that $(X_t)$ is stationary.

Squaring the defining relation of the A(1) process yields $$X_t^2=\phi^2X_{t-1}^2+2\phi X_{t-1}Z_t+Z_t^2$$ hence $$E(X^2)=\phi^2E(X^2)+E(Z^2)$$ that is, $$ E(X^2)=\alpha^2E(Z^2)$$ where $$\alpha^2=\frac1{1-\phi^2} $$ Using the same approach, note that $$X_t=Z_t+\phi X_{t-1}=Z_t+\phi Z_{t-1}+\phi^2X_{t-2}$$ yields $$X_tX_{t-2}=Z_tX_{t-2}+\phi Z_{t-1}X_{t-2}+\phi^2X_{t-2}^2$$ hence $$ E(X_tX_{t-2})=\phi^2E(X^2)=\alpha^2\phi^2E(Z^2). $$ More generally, for every $t$ and $s$,

$$ E(X_tX_s)=\alpha^2\phi^{|t-s|}E(Z^2). $$