I came across an interesting result and a proof for it. I'm confused though about one of the lines. It goes like this.
Using the equality: $$\int_0^sf_r\,dW_r=\int_0^\infty 1_{]0,s]}(r)f_r\,dW_r,\qquad \text{a.s.} $$ we have \begin{align}\mathsf{E}\Bigl[\int_0^sf_r\,dW_r\int_0^tf_r\,dW_r\Bigr] &=\mathsf{E}\Bigl[\Bigl(\int_0^\infty 1_{]0,s]}(r)f_r\,dW_r\Bigr)\Bigl(\int_0^\infty1_{]0,t]}(r)f_r\,dW_r\Bigr)\Bigr]\\ &=\int_0^\infty1_{]0,s]}(r)f_r1_{]0,t]}(r)f_r\,dr=\int_0^\infty1_{]0,s\wedge t]}(r)f^2_r\,dr\\ &=\int_0^{s\wedge t} f^2_r\,dr. \end{align}
My question is, how do they combine the two integrals into one integral on the second last line? I would have thought there should be a double integral? Any help would be much appreciated. Thanks.
We may assume that $s\le t$ and that the $f$ is a convenient step function, where the steps are previsible, and correspond to a division $$ (0=t_0<t_1<t_2<\dots <\underbrace{t_k}_{=s}<\dots<t_N)\ , $$ so that they fit with the cut $s$ of the interval $[0,t]=[0,s)\sqcup[s,t]$, i.e. $s$ is in the list above. Then the integrals are slightly generalized Riemann-Stieltjes sums $$ \begin{aligned} \int_0^s f\; dW &= \sum_{j\ :\ t_j<s} f(t_j)\; (W(t_{j+1})-W(t_j))\ , \\ \int_0^t f\; dW &= \sum_{k\ :\ t_k<t} f(t_k)\; (W(t_{k+1})-W(t_k))\ , \\ \Bbb E\left[ \ \int_0^s f\; dW \int_0^s f\; dW \ \right] &= \Bbb E\sum_{\substack{j\ :\ t_j<s\\k\ :\ t_k<t}} \Big[\ f(t_j)f(t_k)\; (W(t_{j+1})-W(t_j))(W(t_{k+1})-W(t_k))\ \Big] \\ &= \sum_{\substack{j\ :\ t_j<s\\k\ :\ t_k<t}} \Bbb E\Big[\ f(t_j)f(t_k)\; (W(t_{j+1})-W(t_j))(W(t_{k+1})-W(t_k))\ \Big] \\ &=\dots \end{aligned} $$ and now it is essential to understand the next step.
If the involved "time schedules" $t_j$ and $t_k$ differ, then we have either $t_j<t_{j+1}\le t_k< t_{k+1}$, or conversely (with formally exchanged indices $j,k$, and we can build / factorize the mean / expectation $\Bbb E$ above through the filtration step $\mathcal F(t_k)$, where we go through an $\mathcal F(t_k)$--measurable factor times $\Bbb E[W(t_{k+1})-W(t_k)]=0$.
So from the double sum there remain only the "steps" with $j=k$, then these steps are $\le s$, and we have to understand the mean of sumands like $$ f(t_j)^2\; (W(t_{j+1})-W(t_j))^2\ . $$ Then by independence $$ \begin{aligned} \Bbb E&\Big[\ f(t_j)^2\; (W(t_{j+1})-W(t_j))^2\ \Big] \\ = \Bbb E&\Big[\ f(t_j)^2\ \Big]\cdot \Bbb E\Big[\ (W(t_{j+1})-W(t_j))^2\ \Big] \end{aligned} $$ and the second factor leads to $(t_{j+1}-t_j)$, and the whole sum now to a Riemann-Stieltjes integral with this "$dt$", which is exactly what we need.
$\square$