I need to calculate the optimal solution $f(x)$ for the expected absolute loss function, anyone know how to solve it? thank you so much!
let's consider a similar problem first: for squared loss function $$loss(f(x), y) = (f(x) - y)^2$$ the expected loss function would be $$E[loss(f(x),y)] = \int\int (f(x)-y)^2 p(x,y)\ \ dx\ \ dy$$
To minimize the expected loss function, we can do as the following way: $$\int\int (f(x)-y)^2 p(x,y)\ \ dx\ \ dy\\= \int\int\big \{(f(x) - E[y|x])^2+(y-E[y|x])^2 +2(f(x)-E[y|x])(E[y|x]-y)\big\}p(x,y)\ \ dx\ \ dy $$ Since $$\int\int 2(f(x)-E[y|x])~(E[y|x]-y)~p(x,y)\ \ dx\ \ dy = 0$$ it could be proved that when $$f(x)=E[y|x]$$ we will minimize $$E[loss(f(x),y)]$$
now, what if we define? $$loss(f(x),y) = \big |~f(x)-y~ \big |$$
what would be the optimal $f(x)$?
Well, I know the answer now. our goal is to minimize \begin{align*} E[\mathfrak{l}(f(x), y)] &= \int\int |f(x) - y| p(x, y) dx dy\\ &= \int (\int |f(x) - y| p(y|x) dy ) p(x)dx \end{align*} as for every $x$, the value of $f(x)$ could be independently chosen, thus we just need to minimize $$ \int |f(x) - y| p(y|x) dy $$ now calculate the derivative of above expression with respect to $f(x)$, and set it to zero, we have \begin{align*} 0 = \int sign( f(x) - y )~ p(y|x) dy &= \int_{f(x)}^{+\infty} p(y|x) dy - \int^{f(x)}_{-\infty} p(y|x) dy\\ \end{align*} which implies $$ \int_{f(x)}^{+\infty} p(y|x) dy = \int^{f(x)}_{-\infty} p(y|x) dy $$ which is the condition $f(x)$ must satisfy to minimize $E[\mathfrak{l}(f(x), y)]$