Expected amount of time in a state of a CTMC?

396 Views Asked by At

Suppose we have a continuous time Markov chain $X(t)$ on non-negative integers and $P_{i,i+1}=1,\forall i.$ And the exponential rate in state i is $\lambda_i$. Let $T[i,t]$ be the amount of time that the CTMC X spends in state i on $[0,t]$. Prove that $\frac{E[T[i,t]]}{E[T[i-1,t]]}\le\frac{\lambda_{i-1}}{\lambda_i}$.

I am trying to represent $E[T[i,t]]$ first. I think $E[T[i,t]]=\int_0^t sP(T[i,t]=s).$ Therefore it seems we need to find $P(T[i,t]\le s)$ and take derivative. Then I think $P([i,t])$ will be an integrate involving $P(X(z)=i,X(z+s)=i)$. But I am not sure how it relates to our target.

1

There are 1 best solutions below

2
On

I will assume that the initial state is $0$. Here is a proof outline:

Let $t>0$ be arbitrary. Then $T[0,t]$ follows a truncated exponential distribution with mean $\int_0^t \exp(-\lambda_0x)\,dx$, which you can simplify easily. Let $\tau_j$ be the time the chain leaves state $j$ ($j\in\Bbb N_0$), which is a stopping time. Conditional on $\tau_0=s$, $T[1,t]$ follows a truncated exponential distribution with mean $\int_0^{t-s} \exp(-\lambda_1x)\,dx$ if $s<t$, and $T[1,t]=0$ if $s\geq t$. You can then work out $\Bbb E[T[1,t]]=\Bbb E[\Bbb E[T[1,t]\mid\tau_0]]$ explicitly and show that $\Bbb E[T[1,t]]/\Bbb E[T[0,t]]\leq \lambda_0/\lambda_1$ for all $t>0$.

To show that $\Bbb E[T[i,t]]/\Bbb E[T[i-1,t]]\leq \lambda_{i-1}/\lambda_i$ for $i>1$, condition on $\tau_{i-2}$. Conditional on $\tau_{i-2}=s$, $T[i-1,t]$ follows the same distribution as $T[0,t-s]$ and $T[i,t]$ follows the same distribution as $T[1,t-s]$ with $\lambda_{i-1}$ and $\lambda_i$ replacing $\lambda_0$ and $\lambda_1$ respectively, if $s<t$. If $s\geq t$, they're both $0$. So, by the first part, $\lambda_i\Bbb E[T[i,t]\mid \tau_{i-2}=s]\leq \lambda_{i-1}\Bbb E[T[i-1,t]\mid \tau_{i-2}=s]$ for all $s$, and finally $\lambda_i\Bbb E[\Bbb E[T[i,t]\mid \tau_{i-2}]\leq \lambda_{i-1}\Bbb[\Bbb E[T[i-1,t]\mid \tau_{i-2}]]$, which simplifies to give the result.

Anyway, I suspect the first part can be done with less computation.