Calculating the mean return time for this Markov Chain

Question

Given $a > 0$, and a MC with state space consisting of the natural numbers and transition probabilities: $$p(i,i+1) = \frac{1}{a + 1},\,\,\,\,\,\,p(i, i-1) = \frac{a}{a+1},\,\,\,\,\,\,\, i \ge 1$$ and a reflecting barrier at ${0}$ such that $p(0,1) = 1$. How can I compute the mean return time to a state $x$ starting from $x$, i.e. $$E[T_x \mid X_0 = x] \quad \text{ where } \quad T_x = \inf\{n \ge 1 : X_n = x\}$$ I believe I should do this via PGFs but I am not sure how to incorporate the reflecting barrier when calculating the return time.

Answer 1

The mean return time is the reciprocal of the stationary probability, and the stationary distribution can be found by solving the detailed balance equations $$ \pi_i \cdot p(i,i+1) = \pi_{i+1} \cdot p(i+1,i). $$ From these, it is straightforward to get all stationary probabilities in terms of $\pi_0$, and then we can solve for $\pi_0$ by setting the sum of the probabilities equal to $1$.

The rest is just algebra.

Answer 2

This is a single recurrence class (you can reach each state from each other with a non-zero probability)

You may find an stationary distribution $\pi=\{\pi_0,\pi_1,...\}$, in fact for the single recurrence class case it is unique.

Call $\rho=a/(a+1)$ and $1-\rho=1/(a+1)$

Write down the equilibrium equations

$\pi_0=\pi_1\rho$

$\pi_1=\pi_0+\rho\pi_2$ or $\pi_1=\pi_1\rho+\rho\pi_2$ or $\pi_2=(1-\rho)/\rho\pi_1$

$\pi_2=\pi_1(1-\rho )+\rho\pi_3$ or $\pi_2=\pi_2\rho+\rho\pi_3$ or $\pi_3=(1-\rho)/\rho\pi_2=((1-\rho)/\rho)^2\pi_1$

and so on, ($\pi_n=((1-\rho)/\rho)^n\pi_1, n>1$)

Notice that equilibrium probabilities add up to 1 hence

$$\pi_1\{\rho +\sum_{n=0}^{\infty}{((1-\rho)/\rho)^n}\}=1$$

and

$$\pi_1=(2\rho-1)/(2\rho^2)$$

Expected recurrence times are $1/\pi_n$ assuming that equilibrium exists