Probability Markov Chain Distribution

Question

The printing press in a newspaper has the following pattern of break-downs. If it is working today, tomorrow it has a 90% chance of working and a 10% chance of breaking down. If the press is broken today, it has a 60% chance of working tomorrow and a 40% chance of being broken again.

Question: If the press is working today, what are the chances that it is working in 2 days' time?

I spent the last hour trying to figure this out and I give up. At first, I tried using a Markov Chain table that plots the value of current and future states of the press working and breaking down.

The problem is, when I only work with a 2 by 2 table, it is very simple.. but once I need to find the probability of a 3rd day, I simply couldn't settle on any definitive method.

In the end I just tried using the most brute and forced way I could think of..

Day 1 = 100%

Day 2 = 90%

Day 3 = 90% * 1/2

So in total I have (1 + 9/10 + 9/20)/3

Basically, the first day I am told the press is working so its 100%. And since there is a 90% chance of the press working the following day if it is working today, Day 2 then simply has a 90% chance of the press working.

Day 3 is when it gets tricky because in order for the press to be working on day 3, I thought it would need to pass the 90% test twice so I divided 90% by 2 to get 45%.

Lastly, I just averaged the values ( 1 + 9/10 + 9/20)/3.

However, I also thought that since day 1 is given, there is no need to compute it and I only need to calculate day 2 and day 3 which is, (9/10 + 9/20)/2... But I don't know anymore. Please help someone..anyone..

Answer 1

Hint:

The diagram below represents the state transitions in the given system.

Notice that there are two cases that you need to consider:

1. Press working today $\longrightarrow$ press working tomorrow $\longrightarrow$ press working day after tomorrow
2. Press working today $\longrightarrow$ press broken tomorrow $\longrightarrow$ press working day after tomorrow

The probability of the first case is given by $$\mathbb P\left(\mathrm W_0\to\mathrm W_1\to\mathrm W_2\right) = \mathbb P\left(\mathrm W_0\to\mathrm W_1\right)\cdot\mathbb P\left(\mathrm W_1\to\mathrm W_2\right).$$ Similarly, for the second case, we have $$\mathbb P\left(\mathrm W_0\to\mathrm B_1\to\mathrm W_2\right) = \mathbb P\left(\mathrm W_0\to\mathrm B_1\right)\cdot\mathbb P\left(\mathrm B_1\to\mathrm W_2\right).$$

[$\mathrm X_i\to\mathrm Y_j$ represents transitioning from state $\mathrm X$ on the $i^{th}$ day to state $\mathrm Y$ on the $j^{th}$ day]

Answer 2

Let $X_n\in\{U,D\}$ for $n=0,1,2,\ldots$ denote whether the machine is up or down at time $n$. Then $\{X_n:n=0,1,2,\ldots\}$ is a Markov chain with transition matrix $$ P = \begin{pmatrix} \frac9{10}&\frac1{10}\\\frac35&\frac25\end{pmatrix}. $$ The probability in question is: \begin{align} \mathbb P(X_2 = U \mid X_0=U) &= \mathbb P(X_2=U\mid X_1=D)\mathbb P(X_1=D\mid X_0=U) + \mathbb P(X_2=U\mid X_1=U)\mathbb P(X_1=U\mid X_0=U)\\ &=\frac35\cdot\frac1{10} + \left(\frac9{10}\right)^2\\ &= \frac{87}{100}. \end{align}

The $n$-step probabilities $\mathbb P(X_n=j\mid X_0=i)$ are given by $$ P^n = \left( \begin{array}{cc} \frac{1}{7} \left(\left(\frac{3}{10}\right)^n+6\right) & \frac{1}{7} \left(1-\left(\frac{3}{10}\right)^n\right) \\ \frac{6}{7} \left(1-\left(\frac{3}{10}\right)^n\right) & \frac{1}{7}\cdot 10^{-n} \left(2\cdot 3^{n+1}+10^n\right) \\ \end{array} \right). $$ Because this is a finite-state, irreducible, aperiodic Markov chain, it has a unique stationary (and limiting) distribution $\pi_j := \lim_{n\to\infty}\mathbb P(X_n=j)$ independent of the starting state. We compute $\pi$ by taking the limit of $P^n$: $$ \lim_{n\to\infty}P^n = \left( \begin{array}{cc} \frac{6}{7} & \frac{1}{7} \\ \frac{6}{7} & \frac{1}{7} \\ \end{array} \right) $$ So the long-run fraction of time the machine will be working is $6/7$.