Let the homogeneous Markov chain $\left(X_{n}\right)_{n \in \mathbb{N}}$ be described by the following graph:
Then the only missing arrow is from $0$ to $0$ with $\mathbb P\left(X_{1}=0 | X_{0}=0\right)= 1-3/5-1/5 = 1/5$.
Let $k_i$ be the mean time needed to start at $i$ and hit $3$. Then I have the following system of equations:
$$\begin{cases} k_0 &= \frac{1}{5} k_0 + \frac{3}{5} k_1 +1\\ k_1 &= 1k_2 + 1\\ k_2 &= \frac{1}{3} k_1 + \frac{2}{3} k_3 +1\\ k_3 &= 0 \end{cases}$$
Hence $k_1=3,k_2=2,k_0=7/2$
Could you please verify if my argument looks fine or contains logical gaps/errors? Thank you so much for your help!
We can also compute the distribution of the hitting times $k_i$. We have \begin{align} \mathbb P(k_4=k) &= \mathbb P\left(\bigcap_{j=1}^{k-1}\{X_j=0\mid X_0=0\} \right)\mathbb P(X_k=4\mid X_{k-1}=0)\\ &= \left(\frac15\right)^{k-1}\cdot\frac15\\ &= \left(\frac15\right)^k, k=1,2,\ldots \end{align} and \begin{align}\mathbb P(k_4=\infty) &= \mathbb P(X_1=1\mid X_1\ne 0,X_0=0)\\ &= \frac{\mathbb P(X_1=1,X_1\ne 0\mid X_0=0)}{\mathbb P(X_1\ne 1)}\\ &= \frac{3/5}{4/5}\\&=\frac34. \end{align} Clearly $k_5=k_4+1$ a.s. and $k_6=k_5+1$ a.s.
Similarly, $$ \mathbb P(k_1=k) = \left(\frac15\right)^{k-1}\left(\frac35\right), k=1,2,\ldots $$ and $\mathbb P(k_1=\infty) = \frac{1/5}{4/5}=\frac14$. Now clearly $k_2=k_1+1$ a.s. and $$ \mathbb P(k_3 = m+k) = \left(\frac13\right)^{m-2}\left(\frac23\right)\left(\frac15\right)^{k-1}\left(\frac35\right), m=2,3,\ldots,\ k=1,2,\ldots $$