Find the steady state probability that both A and B catch a headache.

Question

I have a question about Markov chain. Let A and B be patients, A has headache at the rate $1$ times/week and recovers from it at rate of $2$ times/week. The patient B has it at the rates $2$ and $4$ /week, respectively. How can I find the steady state prob. that both A and B catch a headache? Thanks.

Answer 1

For each $t>0$ let $$ X(t) = \begin{cases} 0,& \text{Neither patient has a headache at time } t\\ A,& \text{Patient } A \text{ has a headache at time } t\\ B,& \text{Patient } B \text{ has a headache at time } t\\ AB,& \text{Both patients have a headache at time } t. & \end{cases} $$ Then $\{X(t): t\in[0,\infty)\}$ is a continuous-time Markov chain on $S=\{0,A,B,AB\}$ and generator matrix $$G=\begin{bmatrix}-3 & 1 & 2 & 0\\ 2 & -4 & 0 & 2\\ 4 & 0 & -5 & 1\\ 0 & 2 & 4 &-6 \end{bmatrix}, $$ where $$G_{ij}=\begin{cases} \lambda_{ij},& i\ne j\\ -\sum_{j\ne i} \lambda_{ij},& i=j\end{cases}$$ and $\lambda_{ij}$ are the transition rates. Let $P(t)$ be the matrix with entries $\mathbb P(X_t = j\mid X_0= i)$. Then $P(t)$ satisfies the forward Kolmogorov equation $$P'(t) = P(t)G, $$ with solution $$\exp(tG) = \sum_{n=0}^\infty \frac{(tG)^n}{n!}. $$ Since $X$ is irreducible and has a finite state space, it admits a unique stationary distribution. Note that for $s,t>0$ $$P(t+s) = e^{(t+s)G} = e^{tG}e^{sG} = P(t)P(s). $$ So the stationary distribution is given by \begin{align} \lim_{t\to\infty}\lim_{s\to 0}P(t+s)&= \lim_{t\to\infty}\lim_{s\to 0}P(t)P(s)\\ &=\lim_{t\to\infty}P(t)\lim_{s\to0}P(0)\\ &= \lim_{t\to\infty}e^{tG}\lim_{s\to0}e^{tS}\\ &=\lim_{t\to\infty}e^{tG} \end{align} An extremely tedious computation yields the following for $e^{tG}$: $$ \scriptsize\begin{pmatrix} \frac{2\, e^{- 3\, t}}{9} + \frac{2\, e^{- 6\, t}}{9} + \frac{e^{- 9\, t}}{9} + \frac{4}{9} & \frac{e^{- 6\, t}}{9} - \frac{2\, e^{- 3\, t}}{9} - \frac{e^{- 9\, t}}{9} + \frac{2}{9} & \frac{e^{- 3\, t}}{9} - \frac{2\, e^{- 6\, t}}{9} - \frac{e^{- 9\, t}}{9} + \frac{2}{9} & \frac{e^{- 9\, t}}{9} - \frac{e^{- 6\, t}}{9} - \frac{e^{- 3\, t}}{9} + \frac{1}{9}\\ \frac{2\, e^{- 6\, t}}{9} - \frac{4\, e^{- 3\, t}}{9} - \frac{2\, e^{- 9\, t}}{9} + \frac{4}{9} & \frac{4\, e^{- 3\, t}}{9} + \frac{e^{- 6\, t}}{9} + \frac{2\, e^{- 9\, t}}{9} + \frac{2}{9} & \frac{2\, e^{- 9\, t}}{9} - \frac{2\, e^{- 6\, t}}{9} - \frac{2\, e^{- 3\, t}}{9} + \frac{2}{9} & \frac{2\, e^{- 3\, t}}{9} - \frac{e^{- 6\, t}}{9} - \frac{2\, e^{- 9\, t}}{9} + \frac{1}{9}\\ \frac{2\, e^{- 3\, t}}{9} - \frac{4\, e^{- 6\, t}}{9} - \frac{2\, e^{- 9\, t}}{9} + \frac{4}{9} & \frac{2\, e^{- 9\, t}}{9} - \frac{2\, e^{- 6\, t}}{9} - \frac{2\, e^{- 3\, t}}{9} + \frac{2}{9} & \frac{e^{- 3\, t}}{9} + \frac{4\, e^{- 6\, t}}{9} + \frac{2\, e^{- 9\, t}}{9} + \frac{2}{9} & \frac{2\, e^{- 6\, t}}{9} - \frac{e^{- 3\, t}}{9} - \frac{2\, e^{- 9\, t}}{9} + \frac{1}{9}\\ \frac{4\, e^{- 9\, t}}{9} - \frac{4\, e^{- 6\, t}}{9} - \frac{4\, e^{- 3\, t}}{9} + \frac{4}{9} & \frac{4\, e^{- 3\, t}}{9} - \frac{2\, e^{- 6\, t}}{9} - \frac{4\, e^{- 9\, t}}{9} + \frac{2}{9} & \frac{4\, e^{- 6\, t}}{9} - \frac{2\, e^{- 3\, t}}{9} - \frac{4\, e^{- 9\, t}}{9} + \frac{2}{9} & \frac{2\, e^{- 3\, t}}{9} + \frac{2\, e^{- 6\, t}}{9} + \frac{4\, e^{- 9\, t}}{9} + \frac{1}{9} \end{pmatrix} $$ from which it follows that the stationary distribution is $$\pi = \left(\frac49, \frac29, \frac29,\frac19\right). $$ For a practical way to compute $\pi$, observe that because each row of $G$ sums to $0$, there is a nonzero vector $\nu$ such that $\nu G=0$. Solve this equation and normalize so the entries sum to $1$.