Given a Markov-chain, what is the probability of being at a given state?

Question

Given a Markov-chain, what is the probability of being at a given state?

I drew the diagram below just as an example, there is nothing special about it but it would be nice if your answer used it as an example too. In the diagram, $\lambda$ is the rate of arrival, and $\mu$ is the rate of handling—for example like customers at the bank. When there are $3$ customers, service speed doubles, let's say.

In this diagram, what is the probability of being in state 3?

In general, what is the process to determine the probability of being in state $k$?

Answer 1

$$\pi(3)=\frac{\lambda^3}{\lambda^3+6\mu\lambda^2+12\mu^2\lambda+12\mu^3}$$

Answer 2

You can also simply use the time-reversibility equation since it is a birth-death process. These means the rate into state $i$ from $j$ must equal the rate out of state $j$ into state $i$. More formally, $$ \pi_i q_{ij} = \pi_j q_{ji} $$ So, our system of equations are: $$ \begin{align} \lambda \pi_0 &= \mu \pi_1 \\ \tfrac{\lambda}{2}\pi_1 &= \mu \pi_2 \\ \tfrac{\lambda}{3}\pi_2 &= 2\mu \pi_3. \end{align} $$ Lastly, we need our normalizing condition, which means the probabilities of being in each state add up to $1$. More formally, $\sum_i \pi_i = 1$.

Let $\rho = \frac{\lambda}{\mu}$. We want to solve all of our equations so they are in terms of $\pi_0$, so we can then add them together for the normalizing condition. Our equations simplify to $$ \begin{align} \pi_1 &= \rho \pi_0 \\ \pi_2 &= \tfrac{1}{2}\rho^2\pi_0 \\ \pi_3 &= \tfrac{1}{12} \rho^3 \pi_0 \\ \end{align} $$ Now we can apply our normalizing condition to find $\pi_0$: $$ \sum_i \pi_i = 1 \implies \pi_0 + \rho \pi_0 + \tfrac{1}{2}\rho^2\pi_0 + \tfrac{1}{12} \rho^3 \pi_0 = 1 \implies \pi_0 \left(1 + \rho + \tfrac{1}{2}\rho^2 + \tfrac{1}{12}\rho^3 \right) = 1 \\ \implies \pi_0 = \frac{1}{1 + \rho + \tfrac{1}{2}\rho^2 + \tfrac{1}{12}\rho^3} $$ This simplifies to $$ \pi_0 = \frac{12}{\rho^3 + 6\rho^2 + 12\rho + 12} $$ Now, we can plug $\pi_0$ back into our equation for $\pi_3$, which is the probability of being in state three: $$ \pi_3 = \frac{\rho^3}{\rho^3 + 6\rho^2 + 12\rho + 12} $$ I've never heard of the way Did used, but they resolve to the same answer. Hope that helps.