Starting from state 0, the mean number of visits of state 2 before coming back to state 0 is?

Question

The transition probability matrix is$$ \mathrm P= \begin{pmatrix} 0 & 2/3 & 1/3 \\ 1/2 & 0 & 1/2 \\ 1/2 & 1/2 & 0 \\ \end{pmatrix} $$ Starting from state 0, the mean number of visits of state 2 before coming back to state 0 is

1. 5/9
2. 6/9
3. 7/9
4. 8/9
5. None of the above is correct.

If my step is from 0->1->2->0, the probability = $1/6$, but I don't know the total step from state 0 toward itself, and also what is the meaning of "the mean number of visits"?

Answer 1

The "mean number of visits of state $2$" mean how many times do you visit state $2$ on average.Or also the expected number of visit of state $2$. So the mean number of visit is given by the sum, over all runs going from $0$ to $0$, of the probability of the run times the number of times it visit $2$.

So the example you gave ($0\to1\to2\to 0$) gives you that the average number of visit is greater than $1/6*1=1/6$ (it is greater since you only considered one run, you should consider all runs to get the right value.

Denoting $E(n)$ the expected number of visit of state 2 from state $n$, you can right the 3 equation defining $E(0),E(1)$ and $E(2)$ solving this equation will give you the value you look for (it is $E(0)$).

Below are the 3 equations, please do not look at it if you didn't understand something. Ask if something is not clear I'll try to help/be clearer.

$$E(0) = 2/3 E(1)+1/3E(2)$$

$ $

$$E(1) = 1/2 E(2)$$

$ $

$$E(2) = 1 + 1/2 E(1)$$

Answer 2

Let $m_i$ be the mean number of visits to state $2$ before returning to state $0$, starting from state $i$. We want to determine $m_{0}$. By using a one-step analysis, these $m_{i}$ satisfy

\begin{align} m_{0} &= \frac{2}{3} m_{1} + \frac{1}{3} m_{2}, \\ m_{1} &= \frac{1}{2} \cdot 0 + \frac{1}{2} m_{2}, \\ m_{2} &= 1 + \frac{1}{2} \cdot 0 + \frac{1}{2} m_{1}. \end{align}

The interpretation is as follows. Since this is a Markov chain: if I start in state $0$, with probability $\frac{2}{3}$ I transition to state $1$ and can start the process from there and with probability $\frac{1}{3}$ I transition to state $2$ and can start the process from there. The $\frac{1}{2} \cdot 0$ terms indicate that with probability $\frac{1}{2}$ we have returned to state $0$ and the excursion ends. The $1$ term in the $m_2$ expression counts that the process has spent 1 time unit in state $2$.

You can probably solve this system of equations.