Compute the stationary distribution of a Markov Chain on an infinite state space

321 Views Asked by At

I have a Markov Chain on $\mathbb N_0^2$ with a given initial state $(x_0,y_0)$. The allowed transitions for example are of the following form:

$(x,y) \mapsto (x-1,y+2)$ with probability $\propto x$

$(x,y) \mapsto (x+1,y-2)$ with probability $\propto \binom{y}{2}$

$(x,y) \mapsto (x-3,y+5)$ with probability $\propto \binom{x}{3}$

(So in the above example, $x = 0$ or $y = 0$ are absorbing states for $x-1, y-2, x-3$ etc. aren't allowed as we are on $\mathbb N_0^2$.)

I want to compute the stationary distribution of such a Markov Chain.

  1. I need a judgement onto whether there is any hope for closed form solution for problems of the above kind. If yes, please refer me some material to read.
  2. If not, I would want to find the answer using simulation. (My crude idea is that I would start at $(x_0,y_0)$ and watch several random walks. The histograms of the states observed would give me the stationary distribution. However I don't know how long should I be running the random walks. And if I need multiple random walks? So I need some suggestions on what should I be careful about. Also are there better ideas?)

Thanks!