Markov Chain Recurrent State Problem

452 Views Asked by At

The Problem Consider the aging chain on $\{0, 1, 2, \dots\}$ in which for any $n \geq 0$ the individual gets 1 day older from $n$ to $n+1$ with probability $p_n$ but dies and returns to age 0 with probability $1 - p_n$. Find conditions on the $p_n$'s that guarantee that:

(a) 0 is recurrent

My Work

We'll consider first the finite state space $\mathcal{S} = \{0,1,2,\dots,N\}$. In this finite state space, we have irreducibility and aperiodicity, so $\rho_{x0} + \rho_{xN} > 0$ for all $ x \in C = \cal S - \{0,N\}$. So $p(i, i+1) = p_i, p(i, 0) = 1 - p_i$. And for $h(0) = 0, h(N) = 1$, we have \begin{align*} h_N(N - 1) &= p_{N-1} \\ h_N(N-1) &= p_{N-2}p_{N-1} \\ & \vdots \\ h_N(1) &= p_1p_2\cdots p_{N-1} = {\bf P}_1(T_N < T_1) \\ {\bf P}_0(T_0 < T_N) &= p_0(1 - h_N(1)) + (1 - p_0) \\ \xrightarrow{N \to \infty} {\bf P}_0(T_0 < \infty) &= 1 \iff \lim_{N \to \infty}h_N(1) = 0 \end{align*} Thus in order for $ {\bf P}_x(T_0 < \infty) = 1 $, we require that $\prod_{i=1}^\infty p_i = 0$, in words: that the infinite product goes to 0.

(b) 0 is positive recurrent

My Work

In order for 0 to be positive recurrent, we need ${\bf E}_0T_0 < \infty$, where ${\bf E}_0T_0$ is the expected time of first return to 0. So let $\cal S = \{0,1,2, \dots, N\}$, let $C = S - A, A = \{0,N\}$, and set $g(a) = 0 \ \forall a \in A$. So the calculations are} \begin{align*} {\bf E}_0T_0 &= \sum_{k = 0}^\infty k {\bf P}(T_0 = k) \\ &= \sum_{k=0}^\infty k \prod_{i=0}^{k-1} p_i(1 - p_k) \end{align*}

My Question

I am stuck with showing positive recurrence. I am also unsure how to compute ${\bf E}_0T_0$ "directly" as I have been advised for a problem of this sort.

And even if I were show show positive recurrence (which would imply the existence of a stationary distribution $\pi$, how would I do (c) find the stationary distribution?

Thanks for any help.