Can You Help Me With This Continuous Markov Chain Question?

927 Views Asked by At

Consider 2 machines, both of which have an exponential lifetime with mean $\frac{1}{\lambda}$. There is a single repairman that can service machines at an exponential rate $\mu$. Set up the Kolomogorov Backward Equations.

As you recall, the Kolomogorov Backward Equation is the following:

$P_{i,j}'(t) = \sum_{k \neq i} q_{i,k} P_{k,j}(t)-v_{i} P{i,j} (t)$

Since there are two machines, we need three states. One which both machines work, one which one is working and the other needs repair, and the thirs which both machines need repair. I believe this is a birth-death process scenario. I have no idea how to come up with the Markov chain. Also, I do not understand about the second part of the Kolomogorov Backward equation's formula.

Thank you very much. Much help is very appreciated.

1

There are 1 best solutions below

1
On

Define the state space as the number of broken machines $S=\{0,1,2\}$. Now try to draw the graph: three nodes corresponding to the three states and add the transition rates between all of them. You should get a simple birth and death process as you expected. From this graph you can derive the transition matrix of the Markov chain and the appropriate backward equations.