Expectation over Markov Process and discrete Ito integral

132 Views Asked by At

I am doing a research on communication protocol design. A file of $n$ blocks is transferred in several rounds and $R_i$ denotes the number of blocks received in the $i$-th round. The sender sends $n-R_1-R_2-\cdots-R_{i-1}$ blocks in the $i$-th round and $X_i$ denotes the state of the channel in the $i$-th round. Actually, I would like to know the expected value of $\sum_{m=0}^l R_m$ to estimate the transmission rounds.

Assume that $X_i$ satisfies a first-order discrete time-homogeneous Markov chain with two states, thus I can obtain the random variables $R_i\mid R_1,R_2, \ldots, R_{i-1}, X_i$ conditioned on the previous states and the state of the $i$-th step of the Markov process $X$, i.e., $$ R_i \sim Bin(n-R_1-R_2-\cdots-R_{i-1},p(X_i)). $$ By induction on $i$, we have $R_1+R_2+\cdots+R_i \mid X_1,X_2,\ldots,X_i$, i.e., $$ R_1+R_2+\cdots+R_i \sim Bin(n,1-\prod_{m=1}^{i} (1-p(X_i))). $$ I would like to evaluate the expected value of $\sum_{m=1}^l R_m$, and deduce as follows. $$ \begin{aligned} & \mathbb{E}\left[\sum_{m=1}^l R_m \right] \\ = & \mathbb{E}\left[\mathbb{E}\left[\sum_{m=1}^l R_m \Bigg| X_1,X_2,\ldots,X_l \right] \right]\\ = & \mathop{\mathbb{E}}_{\text{over } X_1,X_2,\ldots,X_l} \left[ n\times (1-\prod_{m=1}^{l} (1-p(X_m))) \right]\\ = & n \times \left( 1- \mathop{\mathbb{E}}_{\text{over } X_1,X_2,\ldots,X_l} \left[ \prod_{m=1}^{l}1-p(X_m) \right] \right). \end{aligned} $$ I stuck here because if I carry out the evaluation of the expectations, the number of terms would be $2^l$, which is too large for computation.

An alternative approach is to evaluate the random variable $\sum_{m=1}^l R_m$ using discrete Ito integral, hoping this will simply the formula of the expectation. But I lack the knowledge of real analysis and measure theory, self studying stochastic calculus is difficult for me.

Any suggestions or hints? Thanks in advance!