"Let X(t) be a discrete time Markov chain with state space {0,1,2} and transition matrix \begin{bmatrix}1/3&1/3&1/3\\1/3&1/3&1/3\\1/3&1/3&1/3 \end{bmatrix} Find the expected value of the time it takes the chain to move from state 0 to state 2."
Here is the answer: E0→2 = 1 + (1/3) · 0 + (1/3) · E0→2 +(1/3)·E1→2 = 1+(2/3)·E0→2 = 3.
I find it hard to understand what this means. Is there someone who can explain this clearly to me?
Let $X_0=0$ and $\tau = \inf\{n>0:X_n=2\}$. Observe that $P^2=P$ and hence $P^n=P$ for any positive integer $n$. It follows then that $\tau\sim\mathrm{Geo}\left(\frac13\right)$, that is, $$ \mathbb P(\tau = k) = \frac13\left(\frac23\right)^{k-1},\ k=1,2,\ldots. $$ The expected value of $\tau$ is thus computed by \begin{align} \mathbb E[\tau] &= \sum_{k=1}^\infty k\ \mathbb P(\tau = k)\\ &=\sum_{k=1}^\infty k\ \frac13\left(\frac23\right)^{k-1}\\ &= \frac13\sum_{k=0}^\infty k\left(\frac23\right)^k\\ &=\frac13\left(1-\frac23\right)^{-2}\\ &= 3. \end{align}
More generally, define \begin{align} \tau_0&=\inf\{n\geqslant0:X_n=2\mid X_0=0\}\\ \tau_1&=\inf\{n\geqslant0:X_n=2\mid X_0=1\}\\ \tau_2&=\inf\{n\geqslant0:X_n=2\mid X_0=2\}. \end{align} Clearly $\tau_2=0$, and by the Markov property we have the system of linear equations \begin{align} \tau_0 &= 1 + \frac13\tau_0 + \frac13\tau_1\\ \tau_1 &= 1 + \frac13\tau_0 + \frac13\tau_1, \end{align} which yields $\tau_0=\tau_1=3$.