Iterative re-weighted least squares derivation

77 Views Asked by At

Text states

For a cost function with robust error function $h(e_i)$ $$E(p)=\sum\limits_{i=0}^nh(e_i)$$ it is possible to find an equivalent weighted $L_2$ cost $$E_w(p)=\sum\limits_{i=0}^nw(e_i)\|e_i\|^2,$$ with $$w=\frac{h'(e)}{|e|}$$

In the above, $p$ stands for the parameters of the model we are optimizing. The notation is weird, but I copied it directly from the text I am reading (the error $e_i$ is a function of the parameters and the input).

My question is, how is this derived? Particularly, how is the equality with the weights derived?

The text I am reading is very vague, so I do not know how to go about the derivation. I would appreciate any pointers.

EDIT: My current attempt

Lets say

$$E(x,p,w)=\sum\limits_{i=1}^nw_ie_i(p,x)$$ But we wish to minimize $$E_h(x,p,w)=\sum\limits_{i=1}^nh\circ e_i(p,x)$$ We desire $$\frac{\partial C}{\partial p}=0\iff\frac{\partial C_h}{\partial p}=0$$ Thus $$\frac{\partial }{\partial p}(w_ie_i(x,p))=\frac{\partial }{\partial p}(h\circ e_i(x,p))$$ $$\implies$$ $$w_i\frac{\partial }{\partial p}e_i(x,p)=h'(e_i(x,p))\frac{\partial }{\partial p}e_i(x,p)$$

Thus

$$w_i=h'(e_i(x,p))$$

But this is missing the factor $\frac{1}{|e|}$ the text provides. What am i doing wrong?