Continuous-time Markov Chain with absorbing state: How to add a source term?

48 Views Asked by At

In the situation where you have a continuous-time Markov Chain with rate matrix $M$ and source $\mathbf{b}$ which can be phrased as $$\partial_t \mathbf{p} = M \mathbf{p} + \mathbf{b},$$ you can normally find a solution for $\mathbf{p}$ via an integrating factor; by multiplying both sides by $e^{Mt}$ and working it out, you can obtain $$\mathbf{p}(t) = e^{-Mt}\mathbf{p}_0 + e^{-Mt}(I - e^{-Mt})\mathbf{b}.$$ However, in my case, I have a trapping state in $M$. There is one state in my Markov chain from which there is no escape. This state can be reached from two other states; so $M$ can be decomposed into blocks as $$\begin{bmatrix}M_{11} & \mathbf{m}_{12}\\\mathbf{0} & 1\end{bmatrix},$$ where $\mathbf{m}_{12}$ only has two nonzero elements. Because of this, NumPy complains that this matrix is singular and therefore cannot be inverted. Is there a possibility to still solve this Markov Chain effectively?

Thanks for the help already :)