Consider the transition matrix $p_{xy}=\begin{bmatrix}1 & 0 &0&0\\0.5 & 0 &0.5&0\\0&0.5&0&0.5\\0&0&0&1\end{bmatrix}$
My question though is how to determine the expectation of the first visiting time of the Markov chain of states 1 or 4 from each state, $E(\tau)$.
For states $x=1,4$, it is clear that $E(\tau)=1 $ since it is already there. However for $x=2$ and $3$, it is a little more complicated.
Now I know, $P(X_t ~ \text{visits 1 from 2 any time})= P(X_t ~ \text{visits 4 from 2 any time})=\frac{1}{3}.$
So, when $x=2, E(\tau) = \sum_{t=1}t\mathbb{P}(X_t ~ \text{visits 1 or 4 first}) = ...$ supposedly the solution is 2, but i cannot see how to get this?
One approach is as follows. Given that $X_0 = 2$, the probability for a fixed time $t$ that we land on either $1$ or $4$ for the first time is $(1/2)^t$. With that, we find that the expected time until landing on either $1$ or $4$ is $$ E(\tau) = \sum_{t=1}^\infty t \cdot (1/2)^{t} = 2, $$ which was what we wanted.