product of transition probability and probability of number of arrival

258 Views Asked by At

For a continuous time Markov chain, the transition probability from a state i to a state j in time $t \in \mathbb{R}$ is described by

$p_{t}\left ( i,j \right ) = \sum_{n=0}^{\infty}e^{-\lambda t}\frac{\left ( \lambda t \right )^{n}}{n!}u^{n}\left ( i,j \right )$

$u^{n}\left ( i,j \right )$ is the probability transition for a discrete Markov chain moving from state i to j in n steps.

On the RHS, we see the expression for the probability distribution of the number of arrival by time t and including time t being equal to $n\in [0, \infty)$.

This expression is then multiplied by the transition probability of a discrete Markov chain moving from a state i to j in discrete step $n\in [0, \infty)$

Intuitively, it looks as though the number of arrival at and including some time t in tied to the number of steps a discrete Markov chain takes in moving from state i to j. What is the physical significance of the RHS sans the summation?

1

There are 1 best solutions below

6
On BEST ANSWER

This is only a valid identity assuming that for each state, the sum of transition rates out of that state is $\lambda$.

Making that assumption, transitions will come as a Poisson process: no matter where you are, there is an $\operatorname{Exp}(\lambda)$ time until the next transition. So the total number of transitions in a time interval of length $t$ has the Poisson distribution, $\operatorname{Po}(\lambda t)$. Therefore $e^{-\lambda t} \frac{(\lambda t)^n}{n!}$ is the probability that in a time interval of length $t$, there have been exactly $n$ transitions.

If $u^n(i,j)$ is the probability that after $n$ steps starting at $i$, we end up at $j$, then $e^{-\lambda t} \frac{(\lambda t)^n}{n!}u^n(i,j)$ is the probability that, starting at state $i$, we take exactly $n$ steps in a time interval of length $t$ and end up at state $j$.

Summing over all $n$ means we don't care how many transitions happened over that interval, we just care that we're in state $j$ after $t$ units of time have passed.