When does the conditional expectation of the sum of random variables match with the sum their respective conditional expectations?

Question

I am studying stochastic processes. While studying random walk I acquainted with a notation $N_i$ where $$N_i = \mathrm {Total\ number\ of\ times\ of\ visit\ to\ i}.$$ Let $(X_n)_{n \geq 0}$ be a Markov chain and $S$ be the state space. Then clearly $N_i = \sum\limits_{n=1}^{\infty} \mathbb{1}_{[X_n=i]}$, for some $i \in S$. Now suppose I want to calculate the conditional expectation of $N_j$ given $X_0=i$, for some $i,j \in S$. Let me denote it by $E_i[N_j]$. Then $E_i[N_j]=E[\sum\limits_{n=1}^{\infty} \mathbb{1}_{[X_n=j]}|X_0=i]$. My instructor said that $E[\sum\limits_{n=1}^{\infty} \mathbb{1}_{[X_n=j]}|X_0=i] = \sum\limits_{n=1}^{\infty} E[\mathbb{1}_{[X_n=j]}|X_0=i]=\sum\limits_{n=1}^{\infty} P[X_n=j|X_0=i] =\sum\limits_{n=1}^{\infty} {p_{ij}}^{(n)}$.

Now my question is "Why does the last equality hold"? What would be the reason behind it? Please help me in this regard.

Thank you very much.

Answer 1

The equalty follows from Fubini's theorem. It is true since the terms of the sum are nonnegative.

More details The statement of Fubini's theorem can be found here https://en.wikipedia.org/wiki/Fubini%27s_theorem

Now let $\mu$ be the conditional probability measure on $\Omega$, $$ \mu(A) = P(A|X_0 = i),\qquad \forall A. $$ Let $\nu$ be the counting measure on $\mathbb R_+$, $$ \nu(B) = \text{Card}(B\cap \mathbb N). $$ Let $f(a,\omega) = 1_{[X_{[a]}(\omega)=j]}$ for every $(a,\omega)\in \mathbb R_+\times\Omega$. Then, we have \begin{align} E\bigg[\sum_{n} 1_{[X_{n}(\omega)=j]} \Big| X_0 = i\bigg] & = \int_{\Omega}\bigg[\int_{R_+} f(a,\omega) \nu({\rm d}a)\bigg]\,\mu(d\omega)\\ &= \int_{R_+}\bigg[\int_{\Omega} f(a,\omega)\mu(d\omega) \bigg]\,\nu({\rm d}a)\\ &=\sum_{n} E\bigg[ 1_{[X_{n}(\omega)=j]} \Big| X_0 = i\bigg]. \end{align}

Answer 2

It holds because of the linearity of conditional expectations and the monotone convergence theorem.

The linearity of conditional expectations means that

$$ \mathbb E[X + Y \vert E] = \mathbb E[X\vert E] + \mathbb E[Y \vert E]. $$

Here, $E$ is some event (or even, more generally, a $\sigma$-algebra). This linearity always holds.

We need to appeal to monotone convergence because of the infinite sum. In particular, we have that

\begin{align*} \mathbb E_i[N_j] &= \mathbb E_i \left[\sum_{n=1}^\infty \mathbb 1_{[X_n = j]}\right] \\ & = \mathbb E_i \left[\lim_{m \to \infty} \sum_{n=1}^m \mathbb 1_{[X_n = j]}\right] \\ &= \lim_{m \to \infty}\mathbb E_i \left[\sum_{n=1}^m \mathbb 1_{[X_n = j]}\right] \\ &= \lim_{m \to \infty}\sum_{n=1}^m\mathbb E_i \left[ \mathbb 1_{[X_n = j]}\right] \\ &= \sum_{n=1}^\infty\mathbb E_i \left[ \mathbb 1_{[X_n = j]}\right]. \end{align*}

The third equality is where I appealed to monotone convergence, and the fourth is where I used linearity. The rest are definitions.

An example where we cannot appeal directly to the linearity of conditional expectations is when the number of terms being summed over is random. For example, consider a compound Poisson process $$Y_t = \sum_{i=1}^{N_t} X_i,$$

where $N_t$ is a Poisson process, and the $X_i$'s are iid and independent of $N$. Here, we have that

$$ \mathbb E [Y_t] \neq \sum_{i=1}^{N_t} \mathbb E[X_i], $$

precisely because the number of terms you are summing over is random. The same expression above holds for conditional expectation.