Equation with the expectation of a assessed Markov process

34 Views Asked by At

In my book about Markov processes there is following equation in a proof and I don't see why it's right, I already ask some people in the university, but I had no success, can somebody help me? $$E(\sum_{n=1}^{\infty}e^{-\alpha T_n}\alpha^{-1}(1-e^{-\alpha D_{n+1}})r(Y_n)|Y_0=i) = \frac{1}{\alpha}\int_0^{\infty}e^{-\alpha t}\alpha_i e^{-\alpha_i t}\sum_{j\neq i}q_{ij}V_{\alpha}(j)dt$$

The setting is: $V_{\alpha}(i):=E(\int_0^{\infty}e^{-\alpha t} r(X(t))dt|X(0)=i),\ i\in I$ the expected discounted total gain,$\alpha > 0$, ${X(t),t\geq 0}$ is a Markov Process with a finite state space $I$, $r:I\to \mathbb{R}$ any function, $T_0=0 < T_1 < T_2 < \ldots$ the random changes of states of the process (we consider that we have only finite changes in a finite interval), $D_n:=T_n-T_{n-1}, n\in\mathbb{N}$, $Y_n:=X(T_n)$. ${X(t),t\geq 0} $ and $Y_0,Y_1,\ldots$ both are homogeneous, which means $P(X(t)=j|X(0)=i)=P(X(t+s)=j|X(s)=i)$ for all $t,s \geq 0$, $P(Y_n=j|Y_{n-1}=i)=P(Y_m=j|Y_{m-1}=i)$ for all $n,m\in \mathbb{N}$, one has $P(D_n > t|X(T_{n-1})=i) = e^{-\alpha_i t}$ for a $\alpha_i \geq 0$, independent from n and $q_{ij}:=P(Y_n=j|Y_{n-1}=i)$.

Sorry for those many conditions! And thank you a lot!