Solving the hitting time of a Markov chain using characteristic equation

103 Views Asked by At

Original question: A Markov chain is defined on $S=\{0,1,2,\dots\}$. The transition matrix $P$ is defined this way: $p_{0j}=p_j,\ j\ge 0$ and $p_{i,i+j-1}=p_j,\ i\ge 1,\ j\ge 0$, where $p_j$ is known, $\sum\limits_{j=0}^\infty p_j=1$, $\sum\limits_{j=0}^\infty j\:p_j=\mu$. Then when is $P$ recurrent?

My approach is to solve the hitting probability $h_i := \mathrm{P}_i(\text{hit}\ 0)$. I try to solve $h_i\ (i=0,1,2,\dots)$ in the following equations :

$h_0 = 1$

$h_i = p_0 h_{i-1}+p_1 h_i+p_2h_{i+1}+\dots \quad(i=1,2,3,\dots)$

In order to solve the equations, I treated them like recurrence equations and get the characteristic equation:

$x = p_0+p_1 x+p_2 x^2+\dots=: A(x)$

It is clear that $A(0)=p_0, A(1) = 1, A'(1)=\mu$, and $A(x)$ is monotonically increasing in [0,1]. So when $\mu=1$, $x$ and $A(x)$ intersects at $x=1$. When $\mu\neq 1$, then $x$ and $A(x)$ has two intersection points. Let's say their x-coordinates are 1 and $\lambda$, respectively.

Case 1: $\mu< 1$. Then $\lambda > 1$, $h_i = A+B\lambda^i$, $A$ and $B$ are some constants. Because $0\le h_i\le 1$ and $h_i$ is the minimal solution of the above equations, we have $A=1$, $B=0$, $h_i=1$ for all i, which implies recurrence.

Case 2: $\mu=1$. Then $h_i = A+Bi$, we also have $A=1$, $B=0$, $h_i=1$ for all i, which implies recurrence.

Case 3: $\mu> 1$. Then $\lambda <1$, $h_i = A+B\lambda^i$, $A$ and $B$ are some constants. Because $0\le h_i\le 1$ and $h_i$ is the minimal solution of the above equations, we have $A=0$, $B=1$, $h_i<1$ for all $i>0$, which implies transience.

To sum up, $P$ is recurrent iff $\mu\le1$.

My question is: is it valid to treat such equations like recurrence equations and use the characteristic equation to deduce the solution? I know it works when $A(x)$ only has a finite number of terms, but when it contains an infinite number of terms, I'm not sure. And if it doesn't work, how do I solve the original problem?