I'm doing some self-study and I have encountered a good exercise in understanding how to apply Chapman-Kolmogorov equations to solve various probabilities. Please explain either rigorously or carefully.
A Markov chain $X_{0}, X_{1}, X_{2}$, . . . on the states $\{0$, 1, 2, 3, 4 $\}$ is defined by the transition matrix $$ \mathrm{P}=\left\{\begin{array}{lllll} \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & \frac{1}{5}\\ \frac{2}{5} & \frac{1}{5} & 0 & \frac{1}{5} & \frac{1}{5}\\ 0 & 0 & \frac{1}{2} & \frac{1}{2} & 0\\ 0 & 0 & \frac{1}{2} & \frac{1}{2} & 0\\ 0 & 0 & 0 & 0 & 1 \end{array}\right\} $$ (a) I know that the chain has three classes, $C_{0}=\{0, 1 \}, C_{1}=\{2, 3 \}, C_{2}=\{4\}$ where $C_{0}$ is transient, and $C_{1}$ and $C_{2}$ are closed, and thus, they're recurrent.
(b) Let $T$ be the time until the chain enters one of the closed classes and define $\mu_{i}=\mathrm{E}[T|X_{0}=i]$ for $i\in C_{0}.$
(1) $\displaystyle \mu_{0}=(\mu_{0}+1)\frac{1}{5}+(\mu_{1}+1)\frac{1}{5}+\frac{3}{5}$
(2) $\displaystyle \mu_{1}=(\mu_{0}+1)\frac{2}{5}+(\mu_{1}+1)\frac{1}{5}+\frac{2}{5}$
First, I'm asked to explain why the $\mu_{i}$ satisfy the following two equations. I've tried to, but I have yet to understand why they are as they are. $ \textbf{Please explain this to me}$.
Solving the equations (1) and (2) to obtain $\mu_{0}$ and $\mu_{1}$ only requires simple algebra and I came to the conclusion that $\mu_{0} = \frac{25}{14}$ and $\mu_{1} = \frac{15}{7}$. Again, no idea why.
(c) Let $q_{i}$ be the probability that the chain ends up in state 4 conditional on $X_{0}=i$. Find and explain equations for obtaining $q_{0}$ and $q_{1}$. Solve the equations.
I can see that $q_{0}= P(X_n=4|X_0=0)$, and accordingly $q_{1}= P(X_n=4|X_0=1)$. The process can enter state 4 only if the previous state was in class $C_0$ or $C_2$ itself, since $C_1$ is closed. I've tried to find an equation to show this, but I seem to get stuck. I have to find $P_{04}^n$ and $P_{14}^n$, for sure, but I think I'm missing something by simply assuming that $q_{0}= P_{04}^n$ and $q_{1}= P_{14}^n$. If what I'm thinking is right though, is there a faster way to obtain $P_{04}^n$ and $P_{14}^n$ than having to square the initial matrix over and over, and trying to find a pattern for the corresponding entries?
d) Let $s_{ij}$ denote the expected number of visits to states $j=0$ and $j=1$ conditional on $X_{0}=i$. Find and solve equations for determining the $s_{ij}.$
I have no idea what to do here. What's obvious is that the number of visits to state $j=0$ or $j=1$ has to be the number of transitions from $i=0$ and $i=1$ in to state j, since there can't exist any transitions from a recurrent state to a transient one.