Limiting proportion of time spent in each state of a cyclic birth and death chain

Question

Let $\{X_n\}$ be a birth and death chain on $\mathbb{Z}$.

Let $p_0, p_1, ..., p_m \in (0,1)$ and label state $i$ by $[i]_m = i \text{(mod $m$)}$

Suppose that $p_{i} = $ probability of going from state $[i]_m$ to state $[i+1]_m$

Note that this implies that $1 - p_{i} = $ probability of going from state $[i]_m$ to state $[i-1]_m$

Essentially, what we have is a random walk on the integers with probabilities of going forward $p_i$, and backwards $1-p_i$.

MY QUESTION: I'm interested in finding the limiting proportion of time spent in each state modulo $m$ (each "type of state", as it were).

Now, in the case of the simple random walk (i.e. all the $p_i = 1/2$), the limiting proportion of time spent in state type $[i]_m$ is simply $1/m$. But I'm not sure how to deal with my situation here...

Can anyone help me?

Answer 1

This is a Markov chain on $\{0,1,\ldots,m-1\}$ with transition probabilities $$ P_{ij} = \begin{cases} p_i,& j\equiv i+1\bmod m\\ 1-p_j,& j\equiv i-1\bmod m\\ 0,& \text{ otherwise}. \end{cases} $$ Assuming that $m$ is even so that that the chain is aperiodic, the stationary distribution $\pi$ satisfies $$ \pi_k = \frac{p_{k-1}}{1-p_{k-1}}\pi_{k-1},\ 1\leqslant k\leqslant m-1 $$ which yields the recurrence $$ \pi_k = \pi_0\prod_{i=0}^{k-1}\frac{p_i}{1-p_i}. $$ It follows that $$ \pi_k = \left(\prod_{i=0}^{k-1}\frac{p_i}{1-p_i}\right)\left(\sum_{\ell=0}^{m-1}\prod_{j=0}^{\ell-1}\frac{p_j}{1-p_j}\right)^{-1}. $$

Answer 2

An explicit solution to this problem is given in Section 3 of https://arxiv.org/abs/2001.00291. That section is titled "Stationary distribution of the random walk on the $n$-cycle."

Here is the solution (refer to the link for the derivation). With $q_i:=1-p_i$, define $\Pi_0:=1$ and \begin{align*} \Pi_i&:=\!-\frac{q_0}{q_i}\bigg(\prod_{j=0}^{m-1}\frac{p_j}{q_j}-1\bigg)\bigg(1+\sum_{j=1}^{m-1}\prod_{k=j}^{m-1}\frac{p_k}{q_k}\bigg)^{-1}\bigg(1+\sum_{j=1}^{i-1}\prod_{k=j}^{i-1}\frac{p_k}{q_k}\bigg)+\frac{q_0}{q_i}\prod_{j=0}^{i-1}\frac{p_j}{q_j} \end{align*} for $i=1,2,\ldots,m-1$. Then $$ \pi_i=\frac{\Pi_i}{\Pi_0+\Pi_1+\cdots+\Pi_{m-1}},\quad i=0,1,\ldots,m-1. $$ As a check of the formula, consider the case in which $p_0=p_1=\cdots=p_{m-1}=p\in(0,1)$ and $q_0=q_1=\cdots=q_{m-1}=q:=1-p$. Here the transition matrix is doubly stochastic, so the unique stationary distribution is discrete uniform on $\{0,1,\ldots,m-1\}$. Indeed, algebraic simplification shows that $\Pi_0=\Pi_1=\cdots=\Pi_{m-1}=1$.

A necessary and sufficient condition for reversibility is $$ \prod_{j=0}^{m-1}\frac{p_j}{q_j}=1, $$ in which case the solution above simplifies considerably. In the special case in which $p_0=p_1=\cdots=p_{m-1}=p\in(0,1)$ and $q_0=q_1=\cdots=q_{m-1}=q:=1-p$, reversibility fails unless $p=\frac12$.

It should also be mentioned that the chain is aperiodic if $m$ is odd and has period 2 if $m$ is even. This, however, does not affect existence or uniqueness of the stationary distribution, or the answer to the question.