Visualizing second-order Markov chain

1.2k Views Asked by At

You can visualize a first-order Markov chain as a graph with nodes corresponding to states and edges corresponding to transitions.

Are there any known strategies to visualize a second-order Markov chain, i.e. where transitions have two past elements and one future element?

1

There are 1 best solutions below

0
On

I believe what you mean by 'second order,' is that your state space consists of 2 state vectors, say S1 and S2 (or hypothetically more). Therefore, your transitions are conditioned by both vectors.

However, despite the fact that your transitions are conditioned by both vectors, the transitions are represented by something called the "transition matrix." The transition matrix in the case of a state space consisting of a single vector, the transition matrix is of size S X A, where A is your reward matrix, and each entry in the matrix would have a corresponding transition probability. If you have two state vectors, you combine them into one. So say S1 =[x,y] and S2 = [a,b]. Then your state vector for the entire system, S, is given by S=[ax,ay,bx,by]. And your transition matrix is still represented by a matrix of size S X A.

In short, the visualization of the markov process is no different than if you only had one state vector. It simply expands the the size of your graph by a substantial amount.