Nonhomogenous Chains

45 Views Asked by At

For a discrete-time, nonhomogeneous Markov process, how would you proceed to compute $Pn(t)$ from a given transition matrix $T(t)$?

For homogeneous chains there are two alternative approaches. You either diagonalize $T$, or write the differential equation for the generating function. But for inhomogeneous chains? Diagonalizing won't help, because the transition matrix changes in time, and generating functions meet with obvious complications. So I wonder how you would proceed.

Also, do you know of any case in the literature where explicit formulas have been obtained for $P_n(t)$ in discrete, inhomogeneous Markov chain with more than two states? I'm sure there are many, I'm just looking for examples of the techniques that have been used.

1

There are 1 best solutions below

1
On

Instead of starting with the Markov chain and finding the probabilities $P_n(t)$, consider starting with the probabilities and finding a Markov chain that gives those probabilities.

For a sequence of invertible right stochastic matrices $Q(t)$ with $Q(0) = I$ (where $Q_{ij}(t)$ is to represent the probability of going from state $i$ at time $0$ to state $j$ at time $t$), take the transition matrices $P(t) = Q(t)^{-1} Q(t+1)$. As long as all entries are nonnegative, you have an example.

One way to do this is to start with a homogeneous continuous-time Markov chain $\tilde{Q}(t)$ and let $Q(m) = \tilde{Q}(t_m)$ for some increasing sequence of times $t_m$.