Not sure how to solve or setup this markov chain question

Question

The maternity has 2 beds. Admissions are made at the beginning of the day. With each day, there is a probability of $1/2$ that no admission will arrive and probability $1/2$ that only one potential admission will arrive. A patient can be admitted only if there is an open bed at the beginning of the day. Half of all patients are discharged after one day and all patients that have stayed one day are discharged at the end of the second day.
What is the fraction of days that all beds are utilised?

I have no idea how to start this one up. I especially don't know what the bolded part means. What if there's one patient?

Answer 1

In all Markov-chain problems, the first thing to set up is the state space; in your case the system would be described by pairs which indicate the state of the beds, say $e$ for empty, $f$ for full and $d$ for discharging. So one state would be a pair, for instance $\{e, d\}$ would indicate that one bed is empty and one bed contains a patient that will leave on the next day. One possible state-space describing the clinic is, $$ \mathcal{S} = \big\{\{e,e\}, \{e,d\}, \{e,f\}, \{d,f\}\big\}. $$ Note that in the problem (and in the model) there is no ordering in the beds and $\{e,d\}$ is the same as $\{d,e\}$; note also that the case $\{f,f\}$ is impossible because two guests cannot arrive together and, in case one bed is $f$ on one day it would become $d$ on the following day.

Once the state space is defined, you have to define the transition function which gives the probability of the next state given the current state. In such a small problem can be done by plain enumeration of cases with their associated probabilities $p$:

• $\{e,e\}$: no guest arrives (stay in $\{e,e\}$ with $p=.5$) or one guest arrives (move to $\{e,f\}$, with $p=.5$);
• $\{e,f\}$: Either one guest arrives, and the current guest stays (move to $\{d,f\}$ with $p=.25$); one guest arrives and the current guest leaves (stay in $\{e,f\}$ with $p=.25$); no guest arrives and the current guest stays (move to $\{d,e\}$ with $p=.25$); no guest arrives and the current guest leaves (move to $\{e,e\}$ with $p=.25$).
• $\{e,d\}$: no guest arrives (move to $\{e,e\}$ with $p=.5$); one guest arrives (move to $\{e,f\}$ with $p=.5$).
• $\{d,f\}$: two guests leave (move to $\{e,e\}$ with $p=.5$); one guest leaves (move to $\{e,d\}$ with $p=.5$).

Now, you can draw the transition diagram and set up the transition probability matrix. You can find the stationary probabilities and compute the average time spent in each state.

I hope this helps getting you on track!

Answer 2

The maternity ward contains two beds and admissions are made during beginning of the day. There is $0.5$ probability that an admission will arrive. Half patients get discharged after $1$ and all patients that stay $1$ day get discharged at end of day $2.$

First create the transition probability matrix, the state represents the number of beds empty for $2$ consecutive days.

Here row heads represent number of beds empty yesterday and today, while column heads represent the number of beds empty today and tomorrow. By using a lingo model, the stationary probabilities of the above transition matrix can be determined. The lingo model is as shown below.

In line $2$, a set of states is defined and a steady state probability $PI$ is associated with each state.

In line $3$, for each pairing of states $(I,J)$, a transition probability $TPROB(I,J)$ which equals $P_{ij}$ and mean first passage time time $MFP(I,J)$, from state $I$ to $J$ which equals $M_{ij}$. The transition probabilities are input in lines $6,7,.....,13$ $$PI(J)=\sum_{I}PI(I)\cdot TPROB(I,J)$$ For each $(I,J)$, create equation $$MFP(I,J)=1+\sum_{K\neq j}TPROB(I,J)\cdot MFP(K,J)$$

Stationary probability: Let stationary probability of a state $i$ is given as $PI(ij).$ So from the lingo results, the stationary probabilities are as follows. $$PI(00)=0.182$$

$$PI(01)=0.182$$

$$PI(10)=0.159$$

$$PI(11)=0.250$$

$$PI(12)=0.091$$

$$PI(20)=0.023$$

$$PI(21)=0.068$$

$$PI(22)=0.045$$

Here $PI(00)=0.182$ represents the stationary probability of state $00$.

$a)$

Let the fraction of days where all beds are utilized is represented as $PI(2).$

Here $PI(12)$ denotes the function, where for $2$ consecutive days the $1$ bed was utilized for $1$ day and $2$ beds were in use on the consecutive day.

So, $PI(2)$ can be calculated as:

$$PI(2)=PI(12/2)+PI(20/2)+PI(21/2)+PI(22)$$ $$=0.091/2+0.023/2+0.068/2+0.045$$ $$=0.0455+0.0115+0.034+0.045$$ $$=0.136$$

So, $13.6$% of times, all the beds are utilized.