Let the homogeneous Markov chain $\left(X_{n}\right)_{n \in \mathbb{N}}$ be described by the following graph:
Then the only missing arrow is from $0$ to $0$ with $\mathbb P\left(X_{1}=0 | X_{0}=0\right)= 1-3/5-1/5 = 1/5$.
I would like to compute $h_0^6$, the probability of starting from $0$ and hitting $6$. Let $h_i$ be the the probability of starting from $i$ and hitting $6$. From the diagram, we have $h_1=h_2=h_3 =0$ because $\{1,2,3\}$ is a closed class. Then I have the following system of equations:
$$\begin{aligned} h_0 &= \sum_{i=0}^6 p_{0,i} h_i &&= \frac{1}{5} h_0 + \frac{1}{5} h_4\\ h_4 &= \sum_{i=0}^6 p_{4,i} h_i &&= 1h_5\\ h_5 &= \sum_{i=0}^6 p_{5,i} h_i &&= 1h_6\\ h_6 &= \sum_{i=0}^6 p_{6,i} h_i &&= 1h_4 \end{aligned}$$
It follows that $4 h_0 = h_4 =h_5 =h_6$.
After that, I'm stuck. Could you please shed me some light to finish this exercise? Thank you so much!
You missed using the fact that $h_6 = 1$ (we will reach 6 for sure as we are starting at 6). Once we use that, we get $h_0 = \frac{1}{4}, h_4 = h_5 = h_6 = 1$.