Probability of hitting time as limit

228 Views Asked by At

Given a Markov Chain $(X_n)_{n \ge 0}$, the hitting time of a subset $A$ is defined as:

$$T = \text{inf}\{n \ge 0, X_n \in A \}.$$

Many basic exercises ask to calculate:

$$P(X_{T}=a | X_0=i), \quad a \in A$$

What does it really mean? Is it a limit of a sum of probabilities? That is,

$$\begin{align*} P(X_{T}=a \mid X_0=i) & =P \left(\bigcup_{n \ge1}X_n=a \mid X_0=i \right) \\ &=\sum\limits_{n=1}^{\infty}P(X_n =a, X_{n-1} \notin A, ..., X_1\notin A \mid X_0=i) \end{align*}$$

1

There are 1 best solutions below

0
On

We have $$\{X_T=a\} = \bigcup_{n \in \mathbb{N}_0} (\{X_T =a\} \cap \{T=n\}) = \bigcup_{n \in \mathbb{N}_0} \underbrace{(\{X_n =a\} \cap \{T =n\})}_{=:B_n}.$$ Since the sets $B_n$ are pairwise disjoint, it follows from the $\sigma$-additivity of the probability measure that

$$\begin{align*} \mathbb{P}(X_T =a \mid X_0 = i) &= \mathbb{P} \left( \bigcup_{n \in \mathbb{N}_0} B_n \mid X_0 = i \right) \\&= \sum_{n \in \mathbb{N}_0} \mathbb{P}(B_n \mid X_0 = i) \\ &= \sum_{n \in \mathbb{N}_0} \mathbb{P}(\{X_n=a\} \cap \{T =n\} \mid X_0 = i). \tag{1} \end{align*}$$

It follows from the very definition of the stopping time $T$ that

$$\{T=n\} = \{X_0 \notin A, \ldots, X_{n-1} \notin A, X_n \in A\},$$

and as $a \in A$ this implies

$$\{X_n =a \} \cap \{T=n\} = \{X_0 \notin A, \ldots, X_{n-1} \notin A, X_n=a\}.$$

Plugging this into $(1)$ we conclude that

$$\mathbb{P}(X_T =a \mid X_0 = i) = \sum_{n \in \mathbb{N}_0} \mathbb{P}(X_1 \notin A, \ldots,X_{n-1} \notin A, X_n=a \mid X_0 = i)$$

for any $i \notin A$. (For $i \in A$ the probability on the left-hand side equals $1$, that's the trivial case.)