Let $\lbrace X_{n} \rbrace _{n}$ a Markov Chain and let $P$ be the associated transition matrix $P$= $\begin{pmatrix}.2&.3&.5\\.1&.7&.2\\.5&.3&.2\end{pmatrix}$
Given $ P( X_{0}=0)=.4$ and $ P( X_{0}=1)=P( X_{0}=2)=.3$
Find
$i)$ $E[X_3]$
$ii)$ $P(X_2=0|X_4=1)$
$iii)$ Find $Cov(X,X_3)$
My attempt to $i)$
First we need $P^{(3)}$ =$\begin{pmatrix}.236&.468&.296\\.211&.532&.257\\.263&.468&.269\end{pmatrix}$ then we look for $E[X_3]$
$E[X]=x_1P(X=x_1)+x_2 P(X=x_2)+x_3P(X=x_3)$ in our case
$E[X_3]=0P(X_3=0)+1 P(X_3=1)+2P(X_3=2)=1 P(X_3=1)+2P(X_3=2)$
$E[X_3]= .4P_{01}^{3}+.3P_{11}^{3}+.3P_{21}^{3} +2[.4P_{02}^{3}+.3P_{12}^{3}+.3P_{22}^{3}]$
Am I right?
In case of $ii)$ i was planning to compute $P^{(4)}$ THEN ¿$P(X_2=0|X_4=1)=P_{10}^4$? of maybe try for $P(X_2=0|X_4=1)$= $\frac{P(X_2=0,X_4=1)}{P(X_4=1)}$
And for $iii)$ I don't even understand, I think it was a typo mistake and my teacher wants to refer $Cov(X_i,X_3)$
Any type of help will be appreciated
You are right with case (i). In case (ii) the second formula is valid: $$ \mathbb P(X_2=0\mid X_4 = 1)=\frac{\mathbb P(X_2=0, X_4 = 1)}{\mathbb P(X_4 = 1)}. $$ You know how to find the denominator. For numerator we can use $$ \mathbb P(X_2=0, X_4 = 1) = \mathbb P(X_2=0)\,\mathbb P(X_4=1\mid X_2=0)=\mathbb P(X_2=0) P^{(2)}_{01}. $$ For (iii) it is typo and we can only suspect what $i$ is given. Say, for $$\text{Cov}(X_0, X_3)=\mathbb E[X_0X_3]-\mathbb E[X_0] \mathbb E[X_3]$$ you can repeat calculations from (i). $$ \begin{align} \mathbb E[X_0X_3] =& 1\cdot 1 \cdot \mathbb P(X_0=1,X_3=1) \cr +& 1\cdot 2 \cdot \mathbb P(X_0=1,X_3=2)\cr +& 2\cdot 1 \cdot \mathbb P(X_0=2,X_3=1)\cr +& 2\cdot 2 \cdot \mathbb P(X_0=2,X_3=2) \cr =& \mathbb P(X_0=1)P^{(3)}_{11}+2\mathbb P(X_0=1)P^{(3)}_{12} \cr +&2\mathbb P(X_0=2)P^{(3)}_{21}+4\mathbb P(X_0=2)P^{(3)}_{22}. \end{align} $$