Is this hitting probability counter-intuitive?

54 Views Asked by At

Let the homogeneous Markov chain $\left(X_{n}\right)_{n \in \mathbb{N}}$ be described by the following graph:

enter image description here

Then the only missing arrow is from $0$ to $0$ with $\mathbb P\left(X_{1}=0 | X_{0}=0\right)= 1-3/5-1/5 = 1/5$.

Let $h_i$ be the probability of starting from $i$ and hitting $3$. Then I have the following system of equations:

$$\begin{aligned} h_0 &= \frac{1}{5} h_0 + \frac{3}{5} h_1\\ h_1 &= 1h_2\\ h_2 &= \frac{1}{3} h_1 + \frac{2}{3} h_3\\ h_3 &= 1 \end{aligned}$$

Thus $h_1=h_2=h_3 =1$ and $h_0 = 3/4$.

To me, $h_1 =1$ is quite counter-intuitive because we can go from $1$ to $2$ and vice versa all the time with probability $1/3$. Hence there may be some chance that we start at $1$ and never hit $3$.

Could you please have a check on my attempt? Thank you so much!

1

There are 1 best solutions below

0
On BEST ANSWER

Remember that probability $1$ means an event is "almost sure", not a sure thing. While it's possible to never hit state 3, it is exceedingly unlikely. It is similar to flipping a fair coin and tails never showing up even when flipping the coin indefinitely.