Here is a following example I thought of involving Discrete Time Markov Chains.
Throughout this system, there are two pairs of states : (State A, State B) and (State 1, State 2).
There are 3 stages : Stage 1, Stage 2 and Stage 3. All probabilities across all stages and all chains are unique (i.e. apostrophe ' notation to signify unique).
Remember that all row sums in any matrix must sum to 1 (since these are probabilities).
- Stage 1:
At the start of Stage 1, a Markov Chain tells you the (fixed) probability of transition between State A and State B:
$$P = \begin{bmatrix} P_{aa} & P_{ab} \\ P_{ba} & P_{bb} \end{bmatrix}$$
- Stage 2:
At the start of Stage 2, a Markov Chain tells you the (fixed) probability of transition between State 1 and State 2 if you were in State A at Stage 1 ... and a Markov Chain tells you the (fixed) probability of transition between State 1 and State 2 if you were in State B at Stage 1:
$$P_{|a} = \begin{bmatrix} p_{11} & p_{12} \\ p_{21} & p_{22} \end{bmatrix}$$
$$P_{|b} = \begin{bmatrix} p_{'11} & p_{'12} \\ p_{'21} & p_{'22} \end{bmatrix}$$
- Stage 3:
At the start of Stage 3, a Markov Chain tells you the (fixed) probability of transition between State A and State B if you were in State 1 at Stage 2 ... and a Markov Chain tells you the (fixed) probability of transition between State A and State B if you were in State 2 at Stage 2
$$P_{|1a} = \begin{bmatrix} P_{'aa} & P_{'ab} \\ P_{'ba} & P_{'bb} \end{bmatrix}$$
$$P_{|1b} = \begin{bmatrix} P_{''aa} & P_{''ab} \\ P_{''ba} & P_{''bb} \end{bmatrix}$$
Thus, this system captures a sort of "feedback loop". A person cycles between Stage 1, Stage 2, Stage 3 - and then restarts at Stage 1.
My Question: Is it possible to represent this entire 3 Stage system into a single Markov Chain (i.e. single transition matrix)? Or is this fundamentally impossible?
Thanks!
Note 1: I am thinking I could either model this as a Second Order Markov Chain (i.e. transition probabilities for next state depend on current state and previous state) .... or as Markov Chain where the transition probabilities change with time (e.g. non-homogenous discrete time Markov Chain)?
Note 2: A visualization of this system that I made
