Are there other functions that are similar to Huber loss? More specifically, with the following properties:
- even function
- asymptotically linear
- quadratic close to the origin.
One such function is $\log{(\cosh{(x)})}$, and another is $\sqrt{x^2 + 1}$. Are there any larger families of such functions? Or any useful representations of such functions? Or are there any further categorizations of such functions? Of course you can get new such functions in a trivial way by scaling/shifting horizontally/vertically, but that is not very interesting. This whole question is a bit of a shot in the dark.
Personally, I'm not seeking to find a function to use as a loss function. The reason for my interest is that I have generated data from numerical calculations that have these particular properties, and it just so happens that the data I've generated looks like a loss function. I think going into how the data is generated would be too large of a scope for this question.
Here is the family of functions that I am interested in, for reference.
This is Huber loss.
I think that any such function could be represented as follows. Pick a fixed odd strictly, smooth, increasing function $\phi:[-1,1]\to\mathbb R$. For example $\phi(x) = x/(1-x^2)$, or $\phi(x) = \tan(x/\pi)$, or $f(x) = \tanh^{-1}(x)$, or $f(x) = \text{erf}^{-1}(x)$.
Then any such function can be written as $$ f(x) = a \, \phi(g(\phi^{-1}(x))), $$ where $a > 0$ is the asymptotic slope as $x \to \infty$, and $g:[-1,1] \to [-1,1]$ is an even function.
Now any such $g$ can be approximated by an even standard function, such as a polynomial, or cubic spline. Given $f$, you get $g$ using the formula $$ g(y) = \phi(f(\phi^{-1}(y/a))) ,$$ and then use your favorite method to find the approximation (Taylor's series, interpolating polynomials, Bernstein polynomial, cubic spline, etc, etc).