Given a Markov chain on {0, 1} defined by: \begin{gather} P = \begin{bmatrix} 1-p & p \\ q & 1-q \end{bmatrix} \end{gather} With an initial distribution $\pi(0)$ = [$\frac{q}{p+q}$,$\frac{p}{p+q}$]. I have to find the Expectation $\mathbf{E}[X_n]$, the Variance $\mathbf{V}[X_n]$ and the $\mathbf{Cov}[X_{m+n}, X_n]$.
Intuitively I would say that I can compute the expectation and the variance by multiplying {0,1} with the stationary probabilities. I this correct? How would I factor in the covariance? Isn't this just equal to 0, because the chain converges to a stable distribution i.e. there is no difference between time m=n and n in the long run?
Note that because $\mathbb P(X_n\in\{0,1\})=1$ we have $\mathbb E[X_n] = \mathbb P(X_n=1)$ for all $n\geqslant 1$. Now, \begin{align}\mathbb P(X_n=1) &= \mathbb P(X_n=1\mid X_0=0)\mathbb P(X_0=0) + \mathbb P(X_n=1\mid X_0=1)\mathbb P(X_0=1)\\ &= \mathbb P(X_n=1\mid X_0=0)\frac q{p+q} + \mathbb P(X_n=1\mid X_0=1) \frac p{p+q} \end{align} Computing $P^n$ we have $$ P^n = \left( \begin{array}{cc} \frac{p (-p-q+1)^n+q}{p+q} & \frac{p-p (-p-q+1)^n}{p+q} \\ \frac{q-(-p-q+1)^n q}{p+q} & \frac{q (-p-q+1)^n+p}{p+q} \\ \end{array} \right), $$ so that $$ \mathbb P(X_n=1\mid X_0=0) = \frac{p-p(1-p-q)^n}{p+q} $$ and $$ \mathbb P(X_n=1\mid X_0=1) = \frac{q(1-p-q)^n+p}{p+q}, $$ so that \begin{align} \mathbb E[X_n] &= \mathbb P(X_n=1)\\ &= \frac{p-p(1-p-q)^n}{p+q}\cdot \frac q{p+q} + \frac{q(1-p-q)^n+p}{p+q}\cdot \frac p{p+q}\\ &=\frac p{p+q}. \end{align} Note that this is just $\mathbb P(X_0=1)$. Similarly, we may compute the variance of $X_n$ by the variance of $\pi$: $$ \mathrm{Var}(X_n) = \mathrm{Var}(\pi) = 1^2\mathbb P(X_0=1) = \frac p{p+q}. $$ As for the covariance of $X_{m+n}$ and $X_n$, because the process is stationary, this reduces to the covariance of $X_m$ and $X_0$, which is $$ \mathrm{Cov}(X_m,X_0) = \mathbb E[X_mX_0] - \mathbb E[X_m]\mathbb E[X_0] = \mathbb E[X_mX_0] - \left(\frac p{p+q}\right)^2. $$ Now, \begin{align} \mathbb E[X_mX_0] &= \mathbb E[X_mX_0\mid X_0=0]\mathbb P(X_0=0) + \mathbb E[X_mX_0\mid X_0=1]\mathbb P(X_0=1)\\ &= \mathbb E[X_m]\mathbb P(X_0=1)\\ &= \left(\frac p{p+q}\right)^2, \end{align} so that $\mathrm{Cov}(X_m,X_0)=0$.