How to prove that a Markov chain is transient?

255 Views Asked by At

I have a Markov chain $\{Y_n: n\geqslant 0\}$ where the $Y_n$ are integer-valued.

The probability of going from any state $i$ to its right (i.e., from state $i$ to state $i+1$) is $p$, and the probability of going from any state $i$ to its left (i. e., from state $i$ to state $i-1$) is $1-p$.

Now, I am examining various scenarios pertaining to classifying the states of this Markov chain, for different possible values of $p$.

I can see, by intuition, that for all values of $p$ where $p \ne 1/2$, the Markov chain is transient. This is because when $p > 1/2$, there is a higher probability of moving to the right than to the left, for every state $i$.

This means, starting at $i$, the chain will progressively keep shifting to the right at every time step. So, there is a positive probability that it will not return to $i$, when $p > 1/2$.

Similarly, when $p < 1/2$, starting from $i$, the chain will keep moving to the left at every time step. So again there is a positive probability that it will not return to $i$, when $p < 1/2$.

But I want to be more precise and mathematical when I am proving this property, and not rely on these somewhat informal statements. I'd be grateful for any insight on how to actually prove this.

1

There are 1 best solutions below

3
On BEST ANSWER

Let $N_0 = \inf\{n>0:Y_n=0\mid Y_n=0\}$ be the time until the first return to zero. Let $u_n = \mathbb P(Y_n=0)$ (conditioned on $Y_0=0$) and $t_n=\mathbb P(N_0=n)$. Then $u_0=1$ and $t_0=0$. For $n\geqslant 1$, conditioning on $N_0$ yields \begin{align} u_n &= \mathbb P(Y_n=0)\\ &= \mathbb E[\mathbb P(S_n=0\mid N_0)\\ &= \sum_{k=1}^n \mathbb P(S_n=0\mid N_0=k)\mathbb P(N_0=k)\\ &= \sum_{k=0}^n u_{n-k} t_k. \end{align}

Let $F(s)=\mathbb E[s^{Y_n}]$ be the probability generating function of $N_0$ and $G$ the generating function of the sequence $\{u_n\}$. Then (using Tonelli's theorem to interchange the order of summation): \begin{align} G(s) &= \sum_{n=0}^\infty u_ns^n\\ &= 1+\sum_{n=0}^\infty \sum_{k=0}^n u_{n-k}t_k s^n\\ &= 1 + \sum_{k=0}^\infty u_k z^k\sum_{n=k}^\infty t_{n-k} s^{n-k}\\ &= 1 + F(s)G(s), \end{align} and thus $F(s) = 1-\frac1{G(s)}$. Note that $S_n=0$ precisely when $n=2m$ is even and the number of upward and downward steps are equal. Hence, $q_{2m} = \binom{2m}m(p(1-p))^m$ and the generalised binomial theorem yields \begin{align} G(s) &= \sum_{m=0}^\infty \binom{2m}m(p(1-p))^m s^{2m}\\ &= \sum_{m=0}^\infty \binom{-1/2}m (-1)^m4^m (p(1-p))^m s^{2m}\\ &= \sum_{m=0}^\infty \binom{-1/2}m (-1)^{-1/2-m} (-(4p(1-p)s^2))^m\\ &= \frac1{\sqrt{1-4p(1-p)s^2}}, \end{align} and thus $F(s) = 1-\sqrt{1-4p(1-p)s^2}$.

It follows that $$\mathbb P(N_0<\infty) = F(1) = 1 - \sqrt{1-4p(1-p)}<1,$$ and hence the chain is transient.

Note that if $p=\frac12$ then $F(s)=1-\sqrt{1-s^2}$, so $\mathbb P(N_0<\infty)=1$, but

$$ F'(s) = \frac s{\sqrt{1-s^2}} \stackrel{s\to 1}\longrightarrow \infty, $$ so that $N_0$ has infinite expectation (in this case we say that the chain is null recurrent).