How to prove MAE is convex?

191 Views Asked by At

Consider I have error terms ei in Linear Regression setting. If I want to prove the Convexity of the function, showing the Hessian of that function is 0 should prove the function is convex.

$$f(w_j)=\frac{\left|x_{ij}w_{j}-y_{i}\right|}{n}$$

$$f'(w_j)=\frac{x_{ij}\left(x_{ij}w_{j}-y_{i}\right)}{n\left|x_{ij}w_{j}-y_{i}\right|}$$

$$f''(w_j)=0$$

Since I arrived at 0 it proves the function is convex. Did I do it right?

1

There are 1 best solutions below

0
On BEST ANSWER

The function is not differentiable. Fixing a particular $i$ and let $\lambda \in [0,1]$, we can then use triangle inequality to conclude that

\begin{align} f(\lambda w_{j,1} + (1-\lambda) w_{j,2}) &= \frac{|x_{i,j}(\lambda w_{j,1} + (1-\lambda) w_{j,2})-y_i|}{n} \\ &= \frac{|\lambda (x_{i,j}w_{j,1}-y_i) + (1-\lambda) (x_{i,j}w_{j,2}-y_i)|}{n} \\ &\le \frac{\lambda| (x_{i,j}w_{j,1}-y_i)| + (1-\lambda) |(x_{i,j}w_{j,2}-y_i)|}{n} \\ &= \frac{\lambda f(w_{j,1})+(1-\lambda) f(w_{j,2})}{n} \end{align}

Also, note that sum of convex function is convex.