I have the following transition matrix for a Markov chain:
\begin{bmatrix}1&0&0&0\\1/3&0&1/3&1/3\\1/2&1/2&0&0\\0&0&0&1\end{bmatrix}
I am required to calculate the following $$\Bbb P(X_t\in \{2,3\}|X_0=2).$$ $\textit{Hint: Treat cases of odd and even $t$ separately.}$
Here is my attempt:
The question asks for the probability that we get to either $2$ or $3$ given that we initially start at $2$. If $t\in\{2,4,6,8,...\}$ then the only path that satisfies the required constraints is if we go from $2$ to $3$ and then return back to $2$, so when $t=2$ for example, we have $$p_{2,3}\cdot p_{3,2}=\frac{1}{2}\cdot\frac{1}{3}=\frac{1}{6}.$$ In general, we have, for $t=2k$ $$\Bbb P(X_{2k}\in \{2,3\}|X_0=2)=\Big(\frac{1}{6}\Big)^{k}.$$
If $t\in\{1,3,5,7,...\}$, then we take the path specified in the even case, but finish with an extra step going from $2$ to $3$, so when $t=3$ for example, we have $$p_{2,3}\cdot p_{3,2}\cdot p_{2,3}=\frac{1}{3}\cdot\frac{1}{2}\cdot\frac{1}{3}=\frac{1}{18}.$$ In general, we have, for $t=2k-1$ $$P(X_{2k-1}\in \{2,3\}|X_0=2)=\Big(\frac{1}{3}\Big)^{k}\Big(\frac{1}{2}\Big)^{k-1}.$$
I would include the transition graph for this problem too, but I am not competent enough to code a diagram such as this here.
I have just started a course on Markov chains so apologies in advance for any shaky language such as the "path" from. I know there is a more correct way to communicate what I am saying in my answer using conditional probabilities and the like so feel free to write any modifications in the answer.
Finally then, am I on the right track? Anything that I might have missed or is my attempt good enough as an answer?
Thanks in advance!
It looks excellent and well-explained.
The only mistake/typo I see here is that it should be $$\Bbb P(X_{2k}\in \{2,3\}|X_0=2)=\Big(\frac{1}{6}\Big)^{\color{red}k}$$ for $t=2k$.