I would like to know how to prove that the minimizer of some loss function is equivalent to calculating some properties of the target function? For example:
Regression Loss :
Square loss: $$ f_p(x)= \int_Y ydp(y|x)$$
Absolute loss $$f_(p)=median ~~~p(y|x)$$ where $$median~~ p(.)= y ~~~~~~~s.t. ~~~~ \int_{-\infty}^y tdp(t)= \int_y^{+\infty} tdp(t).$$
Classification Loss : Hinge-loss $$ f_p(x)=sign(p(1|x)-p(-1|x))$$
Logistic loss $$ f_p(x)= \log \frac{ p(1|x)}{p(-1|x)} $$