Understanding the intuition behind this Markov chain model

92 Views Asked by At

An airline reservation system has two computers only one of which is in operation at any given time. A computer may break down on any given day with probability $p$. There is a single repair facility which takes $2$ days to restore a computer to normal. Form a markov chain by taking as states the pairs $(x, y)$ where $x$ is the number of machines in operating condition at the end of a day and $y$ is $1$ if a day's labor has been expended on a machine not yet repaired and $0$ otherwise.

$$ \begin{array}{c c} & \begin{array}{c c } (2,0) & (1,0) & (1,1) & (0,1) \\ \end{array} \\ \begin{array}{c c}(2,0)\\(1,0)\\(1,1)\\(0,1) \end{array} & \left[ \begin{array}{c c} q & p & 0 & 0 \\ 0 & 0 & q & p \\ q & p & 0 & 0 \\ 0 & 1 & 0 & 0\\ \end{array} \right] \end{array} $$

I don't understand what $y$ represents in the phrase, "If a day's labor has been expended on a machine not yet repaired". To me a machine not yet repaired is a broken machine and if the machine is broken then how can you expend a day of labor on it?

Can someone explain the intuition behind this more clearly? For instance, what does $(1,0) \rightarrow (1,1)$ mean in terms of the real physical thing this is modeling?

1

There are 1 best solutions below

1
On BEST ANSWER

A machine can be broken, but the repair facility can fix it. However, it takes 2 days to fix. We need $y$ to keep track of the machines that have broken down and are in the process of being repaired, having been worked on for one day; one more day's repair will get it back in working order. Thus if the state is $(1,0)$, we have one working machine and one broken machine that the repair facility has not yet worked on. It will start now, so that the next day's state will have $y=1$. With probability $q$, the one working machine stays in working order so the next day's state is $(1,1)$. But with probability $p$, the working machine breaks down so the next day's state is $(0,1)$.