Probability of reaching a state before another state - Markov

Question

From the stochastic matrix

$$P = \begin{bmatrix} 0 & 1/2 & 1/2 & 0 & 0 & 0 & 0 \\ 0 & 0 & 1/4 & 3/4 & 0 & 0 & 0 \\ 1/3 & 1/3 & 0 & 1/3 & 0 & 0 & 0 \\ 0 & 0 & 0 & 1/6 & 2/3 & 1/6 & 0 \\ 0 & 0 & 0 & 2/3 & 0 & 0 & 1/3 \\ 0 & 0 & 0 & 1/6 & 0 & 0 & 5/6 \\ 0 & 1/2 & 0 & 0 & 0 & 0 & 1/2 \end{bmatrix}$$

Calculate $\mathbb{P}_4(H^1<H^7)$

I am aware of how to calculate $\mathbb{P}_4(H^1< \infty)$, however unsure on this.

My thinking is that it means the probability starting from state 4, that I 'hit' state 1 before state 7?

Answer 1

Define $u(i)=P_i(H^1<H^7)$, then from the total probability formula based on conditioning on the first step, together with the Markov property and the time homogeneity, you have

$$u(1)=1 \\ u(7)=0 \\ u(i)=\sum_{j \neq i} p_{ij} u(j),i \not \in \{ 1,7 \}.$$

This is a linear system that you can solve.