why does the sum disapear in this derivation: derivation of loss Mean Squared Error. It comes from the following wikipedia page: https://en.wikipedia.org/wiki/Gradient_boosting. It is the last equation of the informal introduction section.
Thank you very much !
Because when differentiating the loss function with respect to $F(x_i)$, you are computing a partial derivative. This means that you're considering the change in the loss function concerning a single instance $F(x_i)$, while keeping all other instances constant.
In the mean squared error (MSE) loss function:
$$L_{MSE}=\frac{1}{n}\Sigma_i^n(y_i-F(x_i))^2$$
when we take the derivative with respect to $F(x_i)$, only the term in the sum that corresponds to that specific $i$ will contribute to the derivative. All other terms in the sum are treated as constants and their derivatives are zero.
So, when we compute
$$\frac{\partial L_{MSE}}{\partial F(x_i)}$$
we are essentially computing the derivative of $(y_i - F(x_i))^2$ with respect to $F(x_i)$, which is $-2(y_i - F(x_i))$. The $1/n$ constant remains so the derivative becomes:
$$\frac{\partial L_{MSE}}{\partial F(x_i)}=\frac{2}{n}(F(x_i)-y_i)$$