Stationary Distribution of a continuous-time Markov Chain

Question

Assume that $d$ particles are distributed over two containers. A particle stays in container $0$ over a random period of time exponentially distributed with parameter $\lambda$, before going to container $1$. Conversely, a particle stays in container $1$ over a random period of time $\sim exp(\mu)$, before going to container $0$. Let $X_t$ denote the amount of particles in container $1$ at time $t$. Note that $(X_t)_{t\geq0}$ is a continuous-time markov chain with state space $I=\{0,...,d\}$. Compute the infinitesimal generator matrix of this chain and find its stationary distribution.

So I found the answer $$\pi_i = \binom{d}{i}(\frac{\lambda}{\mu})^{d+i},$$ but I don't know if it is the correct one because I couldn't prove that $\sum_{i=1}^{d}{\pi_i} = 1$.

Thanks in advance.

Answer 1

\begin{align} \sum_{i=0}^d{d\choose i}\left(\frac{\lambda}{\mu}\right)^{d+i}&= \left(\frac{\lambda}{\mu}\right)^d \sum_{i=0}^d{d\choose i}\left(\frac{\lambda}{\mu}\right)^i\\ &= \left(\frac{\lambda}{\mu}\right)^d\left(1+ \frac{\lambda}{\mu}\right)^d\\ &= \left(\frac{\lambda}{\mu}+ \left(\frac{\lambda}{\mu}\right)^2\right)^d\\ &=1 \end{align} if and only if $\ \frac{\lambda}{\mu} =\frac{\sqrt{5}-1}{2}\ $. Your proposed answer cannot , therefore, be correct. You can also see this from the fact that if $\ Y_t\ $ is the number of particles in container $\ 0\ $ at time $\ t\ $, then, from the symmetry of the problem, the stationary distribution $\ \rho_j\ $ of $\ Y_t\ $ must be given by $$ \rho_j={d\choose j}\left(\frac{\mu}{\lambda}\right)^{d+j}\ . $$ But since $\ Y_t=d-X_t\ $ then we must also have $\ \rho_j=$$\,\pi_{d-j}=$$ {d\choose d-j}\left(\frac{\lambda}{\mu}\right)^{2d-j}\ $ for all $ j\ $, which will only be the case if $\ \lambda=\mu\ $.

Answer 2

Let $\Pi = (\pi_i)_{0 \leq i \leq d}$ be the stationary distribution of the chain. Then: $$\pi_i = \binom{d}{i}\frac{\mu^{d-i}\lambda^i}{(\lambda + \mu)^d}$$ With generator matrix $Q$ given by: $$ q_{ij} = \begin{cases} \mu i & , \quad j = i-1 \\ -\mu i - (d-i)\lambda & , \quad j = i \\ (d-i) \lambda & , \quad j = i+1 \\ 0 & , \quad \text{otherwise.} \end{cases} $$

To prove that, one can use the detailed balance equation for continous-time markov chains:

$$ \pi_i q_{ij} = \pi_j q_{ji}$$

And conclude that:

$$ \pi_{i} = \left(\frac{\lambda}{\mu}\right)^{i}\binom{d}{i}\pi_0 \quad i \neq 0$$

Then you solve for $\pi_0$ using the fact that $\sum_{k=0}^{d}{\pi_i}=1$.