I'm looking for a family of curves that look like a Relu
$$y = \max(0, b(x-a)) \ldots \qquad (1)$$
and looks like this:
but the problem is, sometimes my data saturates, and in that case, I want to fit this curve $$y = \min(\max(0, b(x-a)), h) \ldots \qquad (2)$$
which looks like this:
I came across saturated exponential model,
$$ y = \max\left(0,\ h\left(1\ -\ e^{-b\left(x-a\right)}\right)\right) \ldots\qquad (3) $$
But the problem with this model is, the slope $b$ influences the upper curvature or bend of saturation. Sometimes, my dataset can have low value of $b$ but needs a sharper bend.
You can experiment with the saturated exponential model on desmos: https://www.desmos.com/calculator/4bik4xtp0b
$(1)$ and $(2)$ satisfy exactly what I'm looking for. Is there a way to parametrize them together?
Unfortunately, the Bayesian library I'm using (NumPyro), won't let me do something like this:
$$q * \max(0, b(x-a)) + (1-q) * \min(\max(0, b(x-a)), h) \ldots \qquad (4)$$ where $q$ is drawn from $Bernoulli(p)$ where $p $ prior is set to $U(0,1)$.
So, my question is, is there a way to get something like $(1)$ and $(2)$ together within a single parametric family?
Edit: In case anyone is interested in the NumPyro code, this is my post on NumPyro's forum which includes code for simulating data and fitting the model (along with the error)
https://forum.pyro.ai/t/how-to-fit-models-with-multiple-parametric-families/5080/4
If the range of the first function was $[0,1]$ instead of $[0,\infty)$, then the second function would include the first function as a special case. So let's cheat until we get into that situation! The function $f(x) = e^{-x}$ maps $[0,\infty)$ onto $(0,1]$, and $f^{-1}(x) = -\ln x$ goes back the other way; so we can use the formula $$ y = -\ln \max\bigl\{g,e^{-\max\{0,b(x-a)\}}\bigr\} $$ where $g=0$ gives (1), while $g\in(0,1]$ gives (2) with $h=-\ln g$.
One can use other functions as well if it's helpful—for example, $f(x) = \frac x{x+1}$ and $f^{-1}(x) = \frac x{1-x}$.