For the least squares method why is a factor of 2 taken out when removing the square of the residual? It's probably glaringly obvious but I can't see it.
$$ \begin{eqnarray} R &=& \sum_{i=1}^n(\hat{y}_i-y_i)^2\\ &=& \sum_{i=1}^nr_i^2\\ \Rightarrow \frac{\partial R}{\partial x_j} &=& 2\sum_{i=1}^{n}r_i\frac{\partial r_i}{\partial x_j} \end{eqnarray} $$
Recall the chain rule, i.e., $f(g(x))$, then $$ \frac{\partial}{\partial x} f(g(x)) = f'(g(x)) g'(x), $$ thus $g(x) = x\beta - y$ and $f(x) = x^2$, hence $$ \frac{\partial}{\partial x} f(g(x)) = \frac{\partial}{\partial x} ( x\beta - y) ^2 = f'(g(x)) g'(x) = 2( x\beta -y)x. $$