Derivative of cost functions

Question

We want to compute the derivative of the loss function $J(w)$ with respect to $w,$ where an expression for $J(w)$ is given below.

I am almost clueless about how to solve this problem.Could someone help please?

Answer 1

In this case when the function is of absolute value and therefore not differentiable, I was taught to do multiquadractic approximation of the form $|a| = \sqrt{a^2+k}$, where k is accuracy approximation (usually use $k = 0.0001$).

Therefore, $$J(w) = \frac{1}{2}\sqrt{||Xw-y||^2+k}$$

$$\frac{\partial J(w)}{\partial w} = \frac{X^T(Xw-y)}{2\sqrt{k+||Xw-y||^2}}$$