differential equations for continiuos markov processes

96 Views Asked by At

I'm trying to find the forward equations for birth-and-death processes with no birth, that is, when all $\lambda$ coefficients are zero.

The forward equation for a birth-and-death process has the form:

$$P'_{i0}(s)=\mu_1P_{i1}(s)-\lambda_{0}P_{i0}(s) $$ $$P'_{ij}(s)=\lambda_{j-1}P_{ij-1}(s)+ \mu_{j+1}P_{i,j+1}(s)-(\lambda_{j}+\mu_{j})P_{ij}(s) $$

Reducing to (first line for $i=j$ and second for $j\leqslant i-1$):

$$P'_{ii}(s)=-\mu_iP_{ii}(s)$$ $$P'_{ij}(s)=\mu_{j+1}P_{i,j+1}(s)-\mu_{j}P_{ij}(s) $$

Integrating the first and using $P_{ii}(0)=1$ gives $$P_{ii}(s)=e^{-\mu_{i}s}$$ For $j<i$ $$P'_{ij}(s)+\mu_{j}P_{ij}(s)=\mu_{j+1}P_{i,j+1}(s)$$ $$ e^{\mu_{j}s}[P'_{ij}(s)+\mu_{j}P_{ij}(s)]=e^{\mu_{j}s}[\mu_{j+1}P_{i,j+1}(s)] $$ (multiplied both sides by $e^{\mu_{j}s}$) $$\frac{d}{ds}[e^{\mu_{j}s}P_{ij}(s)]=e^{\mu_{j}s}[\mu_{j+1}P_{i,j+1}(s)]$$ Since $P_{ij}(0)=0$ $$P_{ij}(s)=e^{-\mu_{j}s}*\mu_{j+1}\int_{0}^{s}e^{\mu_{j}t}P_{i,j+1}(t)dt$$ for $j<i$

But this is not sufficient as is it does not allow me to calculate the probabilities recursively.

What am I missing?

1

There are 1 best solutions below

0
On BEST ANSWER

You went almost as far as one can go with this approach...

Assume that all the coefficients $\mu_k$ are different (otherwise things become more complicated to write down). Starting from your recursion relations, one can guess that, for every $0\leqslant j\leqslant i$ there exists some coefficients $(\alpha_{ij}^k)_{j\leqslant k\leqslant i}$ such that, for every $s$, $$ P_{ij}(s)=\sum_{k=j}^i\alpha_{ij}^k\mathrm e^{-\mu_ks}. $$ To compute recursively the array $(\alpha_{ij}^k)_{0\leqslant j\leqslant k\leqslant i}$, one starts from $\alpha_{ii}^i=1$ and one computes the line $(\alpha_{ij}^k)_k$ with $j\leqslant i-1$ from the line $(\alpha_{i,j+1}^k)_k$ using the relations $$ (\mu_j-\mu_k)\alpha_{ij}^k=\mu_{j+1}\alpha_{i,j+1}^k. $$ This yields $\alpha_{ij}^k$ for every $k\geqslant j+1$ and one completes the line $j$ by noting that $P_{ij}(0)=0$ since $j\leqslant i-1$, hence $$ \alpha_{ij}^j=-\sum_{k=j+1}^i\alpha_{ij}^k. $$ Not very easy, right?

Another approach is to work with random variables. To wit, the process jumps from state $i$ to state $i-1$ after a time $T_i$ exponentially distributed with parameter $\mu_i$, hence $$ P_{ij}(s)=P(T_{j+1:i}\leqslant s\lt T_{j:i}),\qquad T_{j:i}=\sum_{k=j}^iT_k, $$ with the conventions that $T_{i+1:i}=0$ and $T_0=+\infty$. Note that the random variables $T_k$ are independent hence this fully determines every $P_{ij}(s)$.

Not very explicit, eh?

So, you see, the choice is between a recursion easy to write down and not so easy to solve, and a probabilistic formulation also easy to write down but not so easy to use... Life is hard. :-)