Soft absolute value

Question

I'm looking for a "soft absolute value" function that is numerically stable. What I mean by that is that the function should have $\mp x$ asymptotes at $\mp\infty$ and behave smoothly in $[-1,1]$.

One easy solution is to integrate $\tanh(x)$, giving $f(x) = \ln(\cosh(x))$, but this formula leads to unnecessary numerical issues for large values of $\|x\|$. Any suggestion greatly appreciated!

Answer 1

Actually integrating the logistic function gives a better result in terms of numerical stability: $$ \int\left( \frac{2}{1+e^{-x}}-1\right) dx = 2J - x + C_1 $$ with $$ J = \int \frac{dx}{1+e^{-x}} = x + \int\frac{-e^{-x}dx}{1+e^{-x}} = x + \ln(1+e^{-x}) + C_2 $$ which put back in the former integral yields $$ f(x) = x + 2\ln\left(\frac{1+e^{-x}}{2}\right) \quad\text{with}\quad f(0)=0 $$

This is only stable for $x\in[0,+\infty)$, but fortunately $f$ is symmetric $$ f(-x) = -x + 2\ln\left( e^x\frac{e^{-x}+1}{2} \right) = f(x) $$ so numeric calculations can be made stable on $(-\infty,0]$.

---

In short, a numerically stable smooth absolute value function is: $$ f(x) = |x| + 2\ln\left(\frac{1+e^{-|x|}}{2}\right) $$

Answer 2

How smooth do you need? Is continuous first derivatives enough? You can fit a quartic through $(-1,1), (0,0), (1,1)$ and demand the proper derivative at $-1,1$. This gives $$y=ax^4+bx^2\\a+b=1\\4a+2b=1\\a=-\frac 12\\b=\frac 32$$ where there is no constant term to go through $(0,0)$ and the function is even because there are no odd order terms. This gives a simple expression.

Answer 3

Depends maybe on what you mean by smooth, but what about $$ \frac{a|x|^3}{1+ax^2}? $$ For $a=10$, it looks like this:

Answer 4

One of the easiest possibilities is to perturb $|x|=\sqrt{x^2}$ a little bit to $$ f_ε(x)=\sqrt{x^2+ε^2} $$ The asymptotics can be derived from $$ 0\le f_ε(x)-|x|=\frac{ε^2}{\sqrt{x^2+ε^2}+|x|}\le \frac{ε^2}{2|x|} $$ If one wants to get a function that has value zero at zero use $$ g_ε(x)=f_ε(x)-ε. $$

Answer 5

I think the simplest is the following:

$$f(x) = x \coth(\alpha x),$$

for some $\alpha$ that you may choose at your convenience. The larger $\alpha$, the closer you are to the true absolute value.

You should be able to resolve any remaining numerical issue at small $x$ with a Taylor expansion.