Discrete markov chain and long run proportions

Question

The Springeld Maternity Ward contains two beds. Admissions are made only at the beginning of the day. Each day, there is a probability $\dfrac12$ that no admission will arrive, and probability $\dfrac12$ that one (and only one) potential admission will arrive. A patient can be admitted only if there is an open bed at the beginning of the day. Half of all patients are discharged after one day, and all patients that have stayed one day are discharged at the end of their second day.

a) What is the fraction of days where all beds are utilised?

b) On the average, what percentage of the beds are utilised?

I have trouble modelling this problem and the states to define. It would be helpful if you can help me get started on this quesiton. Thank you.

Answer 1

For each bed, we need to distinguish between the two states (A) "empty", (B) "occupied by a new patient", and (C) "occupied by a patient who has already been there one day". So the easiest model to find is one with nine states: $$ \{(A,A), (A,B), (A,C), (B,A), (B,B), (B,C), (C,A), (C,B), (C,C)\}. $$ We'd need to make some arbitrary decisions here, such as "if a patient arrives and there are two empty beds, put them in the first one", which means we always transition from $(A,A)$ to $(B,A)$ when this happens and never to $(A,B)$.

To simplify this model, we could combine states that only differ in the order of the beds, since it doesn't matter which bed is which. So we could have six states: $$ \{ AA, AB, AC, BB, BC, CC\}. $$ To figure out the transitions, we can deal with each bed separately: it's important to note that each bed can only change state once each day, since discharges happen after admissions. (The exception is state $AA$, which cannot become $BB$, so here the two beds are not independent.)

For example, if we are in state $AB$:

• The first bed becomes occupied with probability $\frac12$ in the morning, turning into $B$; otherwise it stays $A$.
• The second bed becomes unoccupied with probability $\frac12$ in the evening, turning into $A$; otherwise, it turns into $C$ (the patient is staying for a second day).

This gives us four equally likely possibilities $\{AA, AC, AB, BC\}$.