Discrete-time Markov chain properties

258 Views Asked by At

A Markov chain in discrete time is irreducible, has state space $\{0,1,\dots\}$ and starts at $1$. It is both a branching process and a martingale. Determine the probability of hitting $0$.

1

There are 1 best solutions below

2
On

Under the assumption that this is a birth-and-death reversible MC (rate of inflow=rate of outflow for each state), and the transition probabilities are $p,q,r$ (to go up one state/down one state, remain in the same state) with $p+q+r=1$ and boundary value $h_{0,0}=1$ we can get the following equation: $$ h_{1,0}=qh_{0,0,}+rh_{1,0}+ph_{2,0}\\ p h_{1,0}=qh_{2,0} $$ The second equation come from the reversibility property (detailed balance equation). Solving these, we get: $$ h_{1,0}=\frac{q^2}{p^2-q^2+pq} $$