I want to find transition probabilities of a Markov chain. However, one step probability of state transition is not a fixed but is dependent on the number of users with specific state.
Below is detail:
There are total N users. Each user have a state in a unit time. The state diagram is attached (state diagram.jpeg).
However the state transition probability PS is not a fixed value, but dependent on the number of users with state Ot and Bt. Therefore, I cannot write one step transition probability as $$ P = \begin{pmatrix} 1-Po& 0&Po& 0\\ 0& 1-Pr& 0& Pr\\ Ps& 1-Ps& 0& 0\\ Ps& 1-Ps& 0& 0 \end{pmatrix}, $$where $O, B, Ot, Bt$ states are corresponded to $1, 2, 3, 4$.
Is there any idea to find a transition probability matrix?
Thanks in advance.