I am interested in creating a model in R, where I can implement a non-stationary Markov process.
I would like to create a matrix of probabilities of going from one state to the next during a one year period. I am not familiar with the terms of non-stationary v. stationary when referring to a Markov process. I have only heard of homogeneous and non-homogeneous (which has different implications). After a cursory google search, I still don't have an intuitive feel for non-stationary MC and can't really explain what it means.
It would be great if someone can explain what is meant by a non-stationary Markov chain. An example and simplified explanation would be great.
Intuitively, stationary means that the distribution of the chain at any step is the same. In other words, the chain is in equilibrium, there is no bias coming from an artificial outside source which is not a part of the chain. Here is a simple example.
Let us consider is the following: suppose you have two states A and B. When in A, it stays in A with probability 1/4 and moves to B with probability 3/4. When in B, it stays in B with probability 1/2 and moves to A with probability 1/2. The stationary distribution of such a chain (which you can obtain by solving $\pi P = \pi$, where $P$ is the transition matrix) is $\pi(A) = 2/5$ and $\pi(B) =3/5$. Lets check: If $P(X_0 = A) =2/5$ and $P(X_0 = B ) =3/5 $then $P(X_1 = A) = P(X_1=A | X_0 =A)P(X_0 =A) + P(X_1=A | X_0 =B)P(X_0 =B) = 1/4 . 2/5 + 1/2 . 3/5 =1/10 + 3/10 = 2/5$. So $P(X_1 =B) = 1-P(X_1=A) - 3/5$. Hence $X_1$ has the same distribution as $X_0$ and by induction $X_n$ has the same distribuition as $X_0$. This Markov chain is stationary.
However if we start with the initial distribution $P(X_0 =A)=1$. Then $P(X_1=A) = 1/4$ and hence $X_1$ does not have the same distribution as $X_0$. This chain is not stationary.