Proofs in Stochastic Processes

152 Views Asked by At

Let $X_{n}$ be an irreducible Markov chain on the state space $\{1,\dots,N\}$. Show that there exists $C < \infty$ and $\rho < 1$ such that for any states $i,j$,

$$\mathbb{P} [ X_{m}\neq j , m=0 , \dots, n\mid X_{0}= i]\leq C\rho ^{n}$$

Show that this implies that $\mathbb{E} [T] < \infty$ where $T$ is the first time that the Markov chain reaches the state $j$.

So we know... $\delta > 0$ such that for $i$, the probability of reaching $j$ in the first $N$ steps is greater than $\delta$. I'm not sure how to even use this.

1

There are 1 best solutions below

3
On BEST ANSWER

Hint: The time you avoid state $j$ is heuristically distributed like a geometric random variable. Imagine you had just three states, whose markov chain formed a triangle. Each time you have the possibility of jumping to state $i$, you don't and this happens with probability $1-P(X_{t=1}=i|X_t=j)$. Now generalize this.