I am reading a book for Markov chains and there is a problem (simple) but I cannot understand the average of a product of two random variables in the mc.The problem is:
We have a Markov chain in the state space of $X = \{0,1,2\}$ and transition probability matrix: $$P = \begin{bmatrix} 1/2 & 1/6 & 1/3 \\ 3/4 & 0 & 1/4 \\ 0 & 1/3 &2/3 \end{bmatrix}$$
with initial distribution: $\pi_{0}= (1/2,0,1/2).$
Calculating the $\mathbb{E}(X_0X_1)$ we have :
\begin{align*}
\mathbb{E}(X_0X_1) & = \sum_{x_0=0}^{2}\sum_{x_1=0}^{2} x_0x_1\mathbb{P}(X_0=x_0,X_1=x_1) \\
& = \sum_{x_0=1}^{2}\sum_{x_1=1}^{2} x_0x_1\mathbb{P}(X_0=x_0,X_1=x_1) \\
& = \sum_{x_0=1}^{2}\sum_{x_1=1}^{2} x_0x_1\mathbb{P}(X_0=x_0)\mathbb{P}(X_0=x_0,X_1=x_1) \\
&= \sum_{x_1=1}^{2} 2x_1 \frac{1}{2}\mathbb{P}(X_1=x1 | X_0=2)
\end{align*}
I cannot understand the last two lines.Can someone explain me all the calculations omitted in order to have these two lines ?
There's a typo in the right side of the third line. It should be $$ \sum_{x_0=1}^2\sum_{x_1=1}^2x_0x_1\mathbb{P}\big(X_0=x_0\big)\mathbb{P}\big(X_1=x_1\,\color{red}{\big|}X_0=x_0\big)\ . $$ That this is equal to the expression on the previous line follows from the definition of conditional probability: $$ \mathbb{P}\big(X_1=x_1\,\big|X_0=x_0\big)=\frac{\mathbb{P}\big(X_1=x_1,X_0=x_0\big)}{\mathbb{P}\big(X_0=x_0\big)}\ . $$ The fourth line follows from the facts that $\ \mathbb{P}\big(X_0=1\big)=\pi_0(1)=0\ $, $\ \mathbb{P}\big(X_0=2\big)=\pi_0(2)=\frac{1}{2}\ $, from which it follows that the term in the outer sum for $\ x_0=1\ $ vanishes, and the only remaining term, for $\ x_0=2\ $, is \begin{align} \sum_{x_1=1}^2x_0x_1&\mathbb{P}\big(X_0=x_0\big)\mathbb{P}\big(X_1=x_1\,\big|X_0=x_0\big)\\ &=\sum_{x_1=1}^22x_1\mathbb{P}\big(X_0=2\big)\mathbb{P}\big(X_1=x_1\,\big|X_0=2\big)\\ &=\sum_{x_1=1}^22x_1\pi_0(2)\mathbb{P}\big(X_1=x_1\,\big|X_0=2\big)\\ &=\sum_{x_1=1}^22x_1\frac{1}{2}\mathbb{P}\big(X_1=x_1\,\big|X_0=2\big)\ . \end{align}