Consider a system where two persons sit at a table and share three books. At any point in time both are reading a book, and one book is left on the table. When a person finishes reading his/her current book, he/she swaps it with the book on the table and starts reading. Reading times are exponentially distributed, denote by bi,j the average time for person i to read book j.
Let b = [1 2 4]
[5 1 2]
What is the state space of this markov chain and how can i construct the rate matrix Q ?
I got this exercise from my lecture notes and somehow find the state space confusing since it is a continuous time markov chain.
These are the possible states i could think of :
Person i1 and i2, Book A,B,C
i1,A
i1,B
i1,C
i2,A
i2,B
i2,C
But how can i represent this graphically ? I tried but each user has a separate markov chain(reducible) which i doubt is correct. I think from there constructing the rate matrix based on the rates on matrix b should be ok.