Including non-markovian processes in a birth-death process

Question

Current model

I want to model a stochastic system with a birth-death (Markovian) model. I therefore have this kind of $n$ times $n$ (where $n$ is the number of possible states) transition matrix:

$$\begin{matrix} 1-p & p & 0 & \cdots & 0 & 0 & 0\\ f_{-1}(s,2) & f_{0}(s,i) & f_{+1}(s,i) & \cdots & 0 & 0 & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots \\ 0 & 0 & 0 & \cdots & f_{-1}(s,i) & f_{0}(s,i) & f_{+1}(s,i) \\ 0 & 0 & 0 & \cdots & 0 & p & 1-p \\ \end{matrix}$$

, where $i$ is the current state (the row in the matrices), $s$ is a parameter and $f_{-1}$, $f_{0}$ and $f_{+1}$ are the functions that compute the probabilities of moving to the lower state, remaining in the same state and moving to the higher state respectively. When the model is at one of the two extreme state ($0$ or $n$), it can only leave this state with probability $p$. Therefore, this model has two absorbing states iff p=0. Typically, for what I want to model $p$ is relatively close to $0$ and I expect the most of the time the system will be at one of the two extreme states.

Question

Now, I'd like to extend my model so that the parameter $s$ is drawn from some frequency distribution $F(S=s)$ every time the model leaves (or reach) one of the two extremes states. How can I model such thing? Can I keep using a birth-death markovian model or should necessarily use some other model?

Answer 1

This corresponds to a hidden Markov model $(X_n,S_n)$ where the hidden state $S_n$ determining the transition probabilities from the "non extreme" states is redrawn with the desired distribution $P(S=s)=q(s)$, independently of everything else, each time the observed process $X$ hits the extreme states $0$ and $n$.

Transitions with nonzero probability corresponding to the desired dynamics are the following:

• $(i,s)\to(i,s)$ with probability $f_0(s,i)$, for every $s$, $i\ne0$, $i\ne n$
• $(i,s)\to(i-1,s)$ with probability $f_{-1}(s,i)$, for every $s$, $i\ne0$, $i\ne n$
• $(i,s)\to(i+1,s)$ with probability $f_1(s,i)$, for every $s$, $i\ne 0$, $i\ne n$
• $(0,s)\to(0,s)$ and $(n,s)\to(n,s)$, each with probability $1-p$, for every $s$
• $(0,s)\to(1,t)$ and $(n,s)\to(n-1,t)$, each with probability $pq(t)$, for every $s$ and $t$