$P^{'}(t)=QP(t)=P(t)Q$
These are the Chapman-Kolmogorov backward and forward equations for a countable sate space continuous time Markov Chain. They have the solution,
$ P(t)=e^{Qt} $
I have two questions
1) in the finite state space case, why is this a solution?
2) In the case where we have an infinite state space and the above are the infinite matrices how do we get this as a solution? I know this is a much harder question so maybe just a brief outline of an answer will help. Does the problem come from bringing the derivative inside the infinite sum?
1) This just the solution of this ODE. You can compute it using for instance the definition of the matrix exponential: $$e^A = \sum_{j=0}^\infty \frac{1}{j!}A^j.$$ We have $$P(t) = e^{tQ} = \sum_{j=0}^\infty \frac{1}{j!}t^jQ^j.$$ When differentiating $P(t)$ w.r.t. to $t$, you obtain $$P'(t) = \sum_{j=1}^\infty \frac{1}{j!}(jt^{j-1})Q^j = \left(\sum_{j=1}^\infty \frac{1}{(j-1)!}t^{j-1}Q^{j-1}\right)Q = P(t)Q.$$ Equivalently you can bracket the single $Q$ out to the left and obtain $P'(t) = QP(t)$.
2) I think, when using the definition of the matrix exponential above, you can prove the same statement also for the infinite state space case.