Can the transition probabilities of an inhomogeneous Markov chain be written as an exponential?

1.4k Views Asked by At

If $Z_t$ is a homogeneous continuous-time Markov chain with finite state space $E=\{1,\ldots,p\}$, transition matrices $(P(t))$ and intensity matrix $Q$, it holds that $$ P(t) = \exp(tQ), $$ see for example Chapter 2 of the book "Markov chains" by J. R. Norris. I am interested in whether this can be extended to the inhomogeneous case, and if no, why not. The absence of references indicating any sort of expression for the transition matrices for inhomogeneous chains using matrix exponentials would suggest that a simple extension is not possible.

Before formulating my precise question, let me express a few facts from the theory of inhomogeneous chains. In the in homogeneous case, the intensities are time-dependent. Consider given a continuous function $t\mapsto Q(t)$ of $p\times p$ $Q$-matrices (meaning that the rows sum to one). Assume that we are given a set of transition matrices $P(s,t)$ for $0\le s\le t$ such that $$ \lim_{h\to 0}\frac{1}{h}(P(t,t+h)-I_p) = Q(t) $$ where $I_p$ is the $p\times p$ identity matrix. It then holds that $$ \frac{d}{ds}P(s,t) = -Q(s)P(s,t) $$ and $$ \frac{d}{dt}P(s,t) = P(s,t)Q(t) $$ hold. These are the Kolmogorov backward and forward equations, respectively. This follows from Feller's 1940 paper "On the integro-differential equations of purely discontinuous Markoff processes".

Now consider the mapping $$ \tilde{P}(s,t) = \exp\left(\int_s^t Q(u) du\right). $$ It would be natural to conjecture that $P(s,t) = \tilde{P}(s,t)$. By the matrix chain rule, it at least holds that $$ \frac{d}{ds}\tilde{P}(s,t) = -Q(s)\tilde{P}(s,t), $$ so $\tilde{P}$ satisfies the backward equation. However, by the same reasoning, we appear also to have $$ \frac{d}{dt}\tilde{P}(s,t) = Q(t)\tilde{P}(s,t), $$ which is not the same as the backward equation. The above motivates several questions:

-First, assuming that uniqueness holds for the backward equation (which appears to also have been settled by Feller's 1940 paper, in his Theorem 3), the above would seem to imply $P(s,t) = \tilde{P}(s,t)$. Is this really correct?

-Is it possible to transform the forward equation into the equation solved by the exponential in the above? If yes, how?

-If the equality between $P(s,t)$ and $\tilde{P}(s,t)$ does not hold, where is the error in the above reasoning, and is there some intuitive reason why the result does not hold?

1

There are 1 best solutions below

0
On BEST ANSWER

As explained in the comments, differentiating exponentials of matrices such as $\bar P(s,t)$ is slightly more complicated than in the scalar case since, quite generally, $$\left.\frac{\mathrm d}{\mathrm dt}\mathrm e^{A+tB}\right|_{t=0}=\sum_{i\geqslant0}\sum_{k\geqslant0}\frac{A^iBA^k}{(i+k+1)!},$$ which is neither $B\mathrm e^A$ not $\mathrm e^AB$ in general (that is, except if $A$ and $B$ commute). Thus, if $t\mapsto A(t)$ is differentiable, then $$\frac{\mathrm d}{\mathrm dt}\mathrm e^{A(t)}=\sum_{i\geqslant0}\sum_{k\geqslant0}\frac{A(t)^iA'(t)A(t)^k}{(i+k+1)!}.$$ Finally, if $A(t)$ and $A(s)$ do commute for every $t\ne s$, then $A(t)$ and $A'(t)$ commute for every $t$ and, in this specific case, one gets indeed $$\frac{\mathrm d}{\mathrm dt}\mathrm e^{A(t)}=A'(t)\mathrm e^{A(t)}=\mathrm e^{A(t)}A'(t).$$