Hi I am struggling with a Markov Chain question:
A computer network has two servers, only one of which is in operation at any given time. A server may break down on any given day with probability p. There is a single repairman that requires two days to restore the server to normal. The repairman can only work on one server at a time. A Markov chain is formed by taking as states the pairs (x, y), where x is the number of servers working at the end of a day and y is 1 if a day’s work has been expended on a machine not yet repaired and 0 otherwise.
I understand that: The four possible states are {(2,0),(1,0),(1,1),(0,1)} and The Transition matrix is {{1-p,p,0,0},{0,0,1-p,p},{1-p,p,0,0},{0,1,0,0}}
Show that the probability that the computer network is running is 1/(1+p^2)
I might be making this more difficult than it is, but please help!
They are asking for the probability in the stationary distribution. To find this, let $M$ be the transition matrix and let $I$ be the identity matrx ($1$'s on the diagonal, $0$'s elsewhere). Solve the matrix equation $$(M^T-I)v=0$$ where $M^T$ is the transpose of $M$. This is more commonly shown as $$v(M-I)=0$$ In the first equation $v$ is a column vector, and in the second $v$ is a row vector.
After finding the solutions (there will be infinitely many, parametrized by one variable) normalize the vector so that the sum of the components is $1$. Then the sum of the first three components (or alternatively $1$ minus the fourth component) is the result you are looking for.
For the second problem mentioned in the comments you do the same thing, but the transition matrix will be different.