Expectation of a sample average at a random point

Question

Let $f(x)=\frac{1}{N}\sum_{i=1}^Nf_i(x)$ where $f_i: \mathbb{R} \to \mathbb{R}$ for $i=1,\dots,N$. Let $f_{B}(x)=\frac{1}{|B|}\sum_{i \in B }f_i(x)$ where $B \subseteq \{1,\dots,N\}$ and $g: \mathbb{R}\times \{1,\dots,N\} \to \mathbb{R}$ be a continuous function. Is the following true?

$$ \mathbb{E}_B\Big[f_{B}\Big(g(x, B)\Big)-f\Big(g(x, B)\Big)\Big]=0 $$ If yes, how can we proof it? If not, can we find any relationship between them?

How to sample $B$?

Fix the size of $B$ as $S_B$ and then select $S_B$ elements from $\{1, \dots, N\}$ uniformly without replacement such that $|B|=S_B$.

My try:

Clearly, for any fixed $x$, $$ \mathbb{E}_B\Big[f_{B}\Big(x\Big)-f\Big(x\Big)\Big]=0. $$ However, when we evaluate functions at $g(x, B)$, handling the above is not easy since $g(x, B)$ is a random variable. Can you please help me?

Answer 1

Let me change a little the notation. Let $B=(b_1, b_2, \cdots b_n)$ where $b_i \in \{0, 1\}$. Let $B$ have uniform probability over the set ${\mathcal B} = \{ B : \sum_i b_i = s\}$. Clearly, $E[b_i]=s/n$

Then $f_B(y) = \frac1s \sum_{i=1}^n b_i \, f_i(y)$.

And, indeed, if $y$ is fixed, $E[f_B(y)]= \frac1s \sum_{i=1}^n E[b_i] f_i(y)=\frac1n \sum_{i=1}^n f_i(y)= f(y)$.

But

$$E[f_B(g(x,B))] = \frac1s \sum_{i=1}^n E[b_i \, f_i(g(x,B))] \tag 1$$

And

$$E[f(g(x,B))] = \frac1n \sum_{i=1}^n E[f_i(g(x,B))] \tag 2$$

If we assume that $b_i$ and $f_i(g(x,B))$ are independent (or at least uncorrelated) then we get the desired conclusion.

But, of course, that's a pretty big assumption.

It's difficult to say more without more information about $g(x,B)$

Answer 2

I do not think it is true for any given $g$. For example,with $n=2$, let's define $g$ by :

$$\begin{array}{ccc}g(x, \emptyset) &=& 0 \\ g(x, \{1\}) &=& 0\\ g(x, \{2\}) &=& 1 \\ g(x, \{1,2\}) &=& 0\end{array}$$

As you said in some comment that $S_B$ is a fixed number, I choose for this counter-example $S_B=1$. Then the random variable $B$ has value $\{1\}$ with probability $\dfrac{1}{2}$, and has value $\{2\}$ with probability $\dfrac{1}{2}$.

Then :

$$\begin{array}{lll} \mathbb{E}_B\Big[f_{B}\Big(g(x, B)\Big)-f\Big(g(x, B)\Big)\Big]&=& \dfrac{1}{2}\times\left( f_{\{1\}}\Big(g(x, \{1\})\Big)-f\Big(g(x, \{1\})\Big) \right) \\ &&+\dfrac{1}{2}\times\left( f_{\{2\}}\Big(g(x, \{2\})\Big)-f\Big(g(x, \{2\})\Big)\right)\\ &=& \dfrac{1}{2}\times\left( f_{\{1\}}(0)-f(0) \right) \\ &&+\dfrac{1}{2}\times\left( f_{\{2\}}(1)-f(1)\Big)\right)\\ &=& \dfrac{1}{2}\times\left( f_1(0)-\dfrac{f_1(0)+f_2(0)}{2} \right) +\dfrac{1}{2}\times\left( f_2(1)-\dfrac{f_1(1)+f_2(1)}{2}\Big)\right)\\ &=& \dfrac{f_1(0)-f_2(0)}{4}+\dfrac{-f_1(1)+f_2(1)}{4}\\ \end{array}$$ ans this quantity has absolutely no reason to be equal to $0$ (one can take $f_1(x)=0$ and $f_2(x)=x$ to conclude the counter-example).