I'm looking for an L1 norm loss function. But as many of you well know, absolute value is not differentiable at the origin.
But a good alternative function that approximates the L1 norm that is continuous and differentiable everywhere would be good.
By guess is that I would need to create a piecewise continuous function for some arbitrarily small value epsilon such that
if abs(x) > epsilon, then f(x) = abs(x),
else if abs(x) <= epsilon, then some func(x)
What are some functions you can think of that is continuous and differentiable everywhere and is also a good approximation of an absolute function.
p.s.
now, in terms of the "good approximation" for the continuous piecewise function, I would mean, but not necessarily required, is that the smaller the value of epsilon, smaller the value of some distance between func(x) and abs(x) and that the value of epsilon can get arbitrarily small so that it can be used in wide range of applications where the system could tolerate such small arbitrarily small approximation error.