What values of $p$ make this a transient chain?

1.4k Views Asked by At

Suppose we have a Markov chain with state space $S = \{0, 1, 2, \dots \}$ and probabilities

$p(x, x + 2) = p$, $p(x, x - 1) = 1-p$ for $x > 0$ and $p(0, 2) = p$ and $p(0, 0) = 1 - p$

I would like to find the values of $p$ that make this a transient chain. So I need to find $\alpha(x)$ such that $\inf \alpha(x) = 0$ and

$\alpha(x) = p\alpha(x + 2) + (1-p)\alpha(x - 1)$

I'm having a bit of trouble solving this difference equation. Trying $\alpha(x) = a^x$ doesn't seem to work.

1

There are 1 best solutions below

0
On

For reference, this is Exercise 2.4 from Gregory Lawler's Introduction to Stochastic Processes.

Following Lawler, we define $\alpha(x)$ to be the probability that the process ever visits state 0, starting at $x$. As for any left-continuous random walk, this function is of the form $\alpha(x)=a^x$ where $a$ is the smallest positive root of the equation $z=\sum_{n=0}^\infty p(1,n) z^n$. Here $p(1,n)$ is the transition probability from state $1$ to $n$.

In our case, this equation reads $z=(1-p)+pz^3$ and the solution $a$ will be strictly less than one if and only if $p>1/3$. This is when the chain is transient.