Let $(X,Y)$ be a pair of random variables taking their values from $\mathbb{R}^d, \{0,1\}$.for a Borel-measurable set $A\subset \mathbb{R}^d$, \begin{eqnarray} \mu(A) = \mathbf{P}\{X\in A\} \end{eqnarray} and for any $x\in \mathbb{R}^d$, $\eta(x)$ defined as \begin{eqnarray} \eta(x) = \mathbf{P}(Y=y\vert X=x) \end{eqnarray} see Devroye et al. Then in page 15, it is said that " we have access to nonnegative function $\tilde{\eta}(x)$, $1-\tilde{\eta}(x)$ that approximate $\eta(x)$ and $1-\eta(x)$ respectively."
In my work, I want to define a function $\tilde{\eta}$ such that $\textit{given}$ $\mathcal{D}=\{(x_1, y_1),...,(x_n,y_n)\}$, it approximates $\eta(x)$. How should I define it so that I can use Bayes formulas for conditional probability?
I defined it as $\tilde{\eta}(x,y, \mathcal{D})= p(Y=y\vert X=x, \mathcal{D})$, where by small $p$, I mean an estimate of the conditional probability using $D$ (data). But someone told me that it is not correct.
Define the risk with function $g(x)$ as follows: \begin{eqnarray} L = P[g(x)\ne Y] \end{eqnarray} Then we have \begin{eqnarray} L = \int_{R^d} P[g(x)\ne Y\vert X=x, D_n]\mu(dx) \end{eqnarray} where $D_n$ is data. Then \begin{eqnarray} P[g(x)\ne Y\vert X=x, D_n] &&= 1-P[g(x) = Y\vert X=x, D_n]\cr &&= 1- P[g(x)=1, Y=1\vert X=x, D_n] -P[g(x)=0, Y=0\vert X=x, D_n]\cr &&= 1- I_{[g(x)=1]} P[Y=1\vert X=x, D_n] -I_{[g(x)=0]} P[Y=0\vert X=x, D_n] \end{eqnarray} So we can define \begin{eqnarray} \bar{\eta}(x,y,D) = P[Y=1\vert X=x, D] \end{eqnarray}