I'm struggling with this question. It's like a Nonhomogeneous Poisson Process applied to Markov Chains.
I have a Markov Chain composed of three states (1,2,3) as follows.
\begin{bmatrix}0.5&0.3&0.2\\0&0.6&0.4\\0&0&1\end{bmatrix}
The number of error messages I get in each state is a Poisson Process. The mean of the number of error messages I get per minute is 0, 1, and 2 for states 1, 2, 3, respectively.
What is the probability that I will have accumulated 4 error messages when the 5th minute finishes given that the initial state is 1?
Or in general, what is the probability that I have a total of k messages at the end of nth minute given the initial state is s? How can I generalize it for large n given the initial state s?
This is a Markov-modulated Poisson process. Let $\{X_n:n=0,1,\ldots\}$ be the underlying Markov chain and $N(t)$ the arrival process. Then $N(t) = N_1(t)\mathsf 1_{\{2\}}(X_n) + N_2(t)\mathsf 1_{\{3\}}(X_n)$ where $N_1(t)$ is a Poisson process with rate $1$ and $N_2(t)$ a Poisson process with rate $2$.
Assume $X_0=1$. Let $\tau_i = \inf\{n>0:X_n=i\}$, $i=2,3$. Then $$ \mathbb P(\tau_2=i) =\frac3{10}\left(\frac12\right)^{i-1},\ i=1,2,\ldots $$ and $$ \mathbb P(\tau_3=j) = \begin{cases} \frac15,& j=1\\ \frac{11}{50},& j=2\\ \frac{91}{500},& j=3\\ \frac{851}{5000},& j=4 \end{cases} $$ We have \begin{align} \mathbb P(N(5)=m) &= \mathbb P(\tau_3=1)\mathbb P(N_2(5)-N_2(1)=m)+\sum_{i=1}^4\mathbb P(\tau_2=i)\mathbb P(\tau_3\geqslant 5)\mathbb P(N_1(5)-N_1(i)=m)+\sum_{i=1}^4\sum_{j=i+1}^4\mathbb P(\tau_2=i)\mathbb P(\tau_3=j)\sum_{m_1=0}^m \mathbb P(N_1(j)-N_1(i)=m_1)\sum_{m_2=0}^{m-m_1}\mathbb P(N_2(5)-N_2(j)=m_2). \end{align} Unfortunately, lack of a closed form for $\mathbb P(\tau_3=j)$ makes this very tedious to evaluate. But I believe the idea is correct.