MLE for CTMC parameters

264 Views Asked by At

Let the data set be $$D = \{(s_0, t_0), (s_1, t_1), ..., (s_{N-1}, t_{N-1})\}$$ where $N=|D|$. Each $s_i$ is a state from the state space $S$ and during the time $[t_i,t_{i+1}]$ the chain is in state $s_i$. Let the parameters be represented by $$\theta = \{\lambda, P\}$$ where $\lambda$ is the vector of holding parameters for each state and $P$ the transition matrix of the embedded discrete time Markov chain.

Then the probability is given by $$Pr(D | \theta) \propto \lambda(s_0)e^{-\lambda(s_0)(t_1-t_0)}Pr(s_1|s_0) \;.\; \lambda(s_1)e^{-\lambda(s_1)(t_2-t_1)}Pr(s_2|s_1)\; ... \;\lambda(s_{N-2})e^{-\lambda(s_{N-2})(t_{N-1}-t_{N-2})}Pr(s_{N-1}|s_{N-2}) $$

Let $n(j|i)$ denote the number of $i$->$j$ transitions in $D$, and $n(i)$ the number of times $s_i$ occurs in $D$. Let $t(s_i)$ denote the total time the chain spends in state $s_i$.

Then the MLEs are given by $$\lambda(s) = \frac{n(s)}{t(s)},Pr(j|i)=\frac{n(j|i)}{n(i)}$$

I hope my calculation is correct so far. I am new to the theory of CTMCs, and if I understand correctly, the holding parameters for each state and the transition parameters are independent, which enables us to perform maximum likelihood estimation independently for each set, thus resulting in the above equations. Is this correct? Any hint would be great should my method be incorrect.