Bayes Risk problem: prove $\lim_{n\rightarrow +\infty}\frac{\mathbb{E}(L_n)-L^*}{\sqrt{\mathbb{E}( ( \eta_n(X)-\eta(X) )^2 )}}=0$

77 Views Asked by At

The object of the proof is to show that $L_n-L^*$ converges to $0$ faster than the error $L_2$, the conditional expectation error. $L^*$ denotes Bayes' risk. The function $\eta$ is given by $\eta(x)=\mathbb{E}(Y|X=x)$ and $L_n$ is the empirical prediction error.

Prove $$\lim_{n\rightarrow +\infty}\frac{\mathbb{E}(L_n)-L^*}{\sqrt{\mathbb{E}( ( \eta_n(X)-\eta(X) )^2 )}}=0$$

if $\eta_n$ verifies $\lim_{n\rightarrow\infty} \mathbb{E}( ( \eta_n(X)-\eta(X) )^2 )=0$

The idea might be to use $g_n(x)=1_{\{\eta_n(x)>1/2\}}$ because now $\mathbb{E}(L_n)-L^*=2\mathbb{E}(|n(X)-1/2|1_{\{g(X)\neq g^*(X)\}})$. Now I want to show that $\mathbb{E}(L_n)-L^*=2\mathbb{E}(|n(X)-1/2|1_{\{g(X)\neq g^*(X)\}})$ can be bounded, this is, I want to prove there is an $\epsilon$ such such that $$ \mathbb{E}(|n(X)-1/2|1_{\{g(X)\neq g^*(X)\}})\leq \mathbb{E}(|n(X)-\eta_n(X)|1_{\{g(X)\neq g^*(X)\}}1_{|\eta(X)-1/2|\leq \epsilon}1_{\eta(X)\neq1/2}) + \mathbb{E}(|n(X)-\eta_n(X)|1_{\{g(X)\neq g^*(X)\}}1_{|\eta(X)-1/2|> \epsilon})$$.

If the latter can proved, and it is bounded, taking limits would show the limit of the problem is zero.