I conjecture that for any $\beta>0$, \begin{align} \sigma(\beta) > \Phi(x_{\beta}) \quad\text{where}\quad x_{\beta} = \beta \phi(x_{\beta}). \end{align}
- $\sigma$ denotes the sigmoid function defined by $\sigma(x) = \frac{1}{1+e^{-x}}$.
- $\Phi$ and $\phi$ denote the standard normal cdf and pdf, respectively.
The conjecture seems to be true according to my numerical analysis, but I am unsure about a formal proof.
I would appreciate it if you would share your thoughts or suggestions. Thank you.
We rephrase the problem as follows.
Problem. Prove that, for all $\beta > 0$, $$\frac{1}{1 + \mathrm{e}^{-\beta}} > \int_{-\infty}^{u} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t$$ where $u$ is a real number satisfying $$u = \beta \cdot \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-u^2/2}.$$ (Note: Clearly, such $u$ exists (unique) and is positive.)
$\phantom{2}$
Proof.
We split into two cases.
Case 1. $\beta \ge 3$
Let $f(u) := u - \beta \cdot \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-u^2/2}$. We have $f'(u) > 0$. We have $$f(\sqrt{2\ln \beta}) = \sqrt{2\ln \beta} - \frac{1}{\sqrt{2\pi}} \ge \sqrt{2\ln 3} - \frac{1}{\sqrt{2\pi}} > 0 = f(u)$$ which results in $\sqrt{2\ln \beta} \ge u$. It suffices to prove that $$\frac{1}{1 + \mathrm{e}^{-\beta}} > \int_{-\infty}^{\sqrt{2\ln \beta}} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t. \tag{1}$$
We have, for all $u>0$ $$1 - \frac{u^2 - 1}{u^3\sqrt{2\pi}}\,\mathrm{e}^{-u^2/2}\ge \int_{-\infty}^{u} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t. \tag{2}$$ (Proof: Let $g(u) := \mathrm{LHS} - \mathrm{RHS}$. We have $g'(u) = - \frac{3}{u^4\sqrt{2\pi}}\mathrm{e}^{-u^2/2}< 0$ for all $u > 0$. Also, $g(\infty) = 0$. Thus, $g(u)\ge 0$ for all $u > 0$.)
From (1) and (2), it suffices to prove that $$\frac{1}{1 + \mathrm{e}^{-\beta}} > 1 - \frac{2\ln \beta - 1}{\sqrt{2\ln \beta}^3\sqrt{2\pi}}\cdot \frac{1}{\beta}. \tag{3}$$
We have \begin{align*} &\mathrm{e}^{-\beta} = \frac{1}{\mathrm{e}^3\mathrm{e}^{\beta-3}} \le \frac{1}{\mathrm{e}^3[1 + (\beta - 3) + \frac12(\beta - 3)^2]}, \tag{4}\\[6pt] &\frac{2\ln \beta - 1}{\sqrt{2\ln \beta}^3\sqrt{2\pi}} = \frac{1 - 1/(2\ln\beta)}{\sqrt{2\ln \beta}\sqrt{2\pi}} \ge \frac{1 - 1/(2\ln 3)}{\sqrt{2\ln \beta}\sqrt{2\pi}},\tag{5}\\[6pt] &\ln \beta = \ln 3 + \ln (1 + (\beta - 3)/3) \le \ln 3 + (\beta - 3)/3. \tag{6} \end{align*}
From (3)-(6), it suffices to prove that $$\frac{1}{1 + \frac{1}{\mathrm{e}^3[1 + (\beta - 3) + \frac12(\beta - 3)^2]}} > 1 - \frac{1 - 1/(2\ln 3)}{\sqrt{2\ln 3 + 2(\beta - 3)/3}\cdot \sqrt{2\pi}}\cdot \frac{1}{\beta},$$ or $$\frac{1 - 1/(2\ln 3)}{\sqrt{2\ln 3 + 2(\beta - 3)/3}\cdot \sqrt{2\pi}}\cdot \frac{1}{\beta} > \frac{2}{\mathrm{e}^3(\beta^2 - 4\beta + 5) + 2},$$ or $$\frac{(1 - 1/(2\ln 3))^2}{[2\ln 3 + 2(\beta - 3)/3]\cdot 2\pi}\cdot \frac{1}{\beta^2} > \frac{4}{[\mathrm{e}^3(\beta^2 - 4\beta + 5) + 2]^2},$$ or (using $\mathrm{e}^3 > 20$ and $2\ln 3 > 2 + 1/6$ and $2\ln 3 < 2 + 1/5$ and $2\pi < 6 + 1/3$) $$\frac{(1 - 1/(2 + 1/6))^2}{[(2 + 1/5) + 2(\beta - 3)/3]\cdot (6 + 1/3)}\cdot \frac{1}{\beta^2} > \frac{4}{[20(\beta^2 - 4\beta + 5) + 2]^2},$$ or $${\frac {1}{3211}}\,{\frac {220500\,{\beta}^{4}-1796110\,{\beta}^{3}+5767467\,{ \beta}^{2}-8996400\,\beta+5735205}{ \left( 3+10\,\beta \right) {\beta}^{2} \left( 10\, {\beta}^{2}-40\,\beta+51 \right) ^{2}}} > 0$$ which is true.
$\phantom{2}$
Case 2. $0 < \beta < 3$
Let $f(u) := u - \beta \cdot \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-u^2/2}$. We have $f'(u) > 0$. Using $\sqrt{2\pi} > 5/2$ and $\mathrm{e}^{-2\beta^2/25} = \frac{1}{\mathrm{e}^{2\beta^2/25}} \le \frac{1}{1+ 2\beta^2/25}$, we have $$f(2\beta/5) = \frac{2\beta}{5} - \frac{\beta \mathrm{e}^{-2\beta^2/25}}{\sqrt{2\pi}} \ge \frac{2\beta}{5} - \frac{ \beta \cdot \frac{2}{5}}{1+ 2\beta^2/25} = \frac{4\beta^3}{10\beta^2 + 125} > 0 = f(u)$$ which results in $2\beta/5 \ge u$. It suffices to prove that $$\frac{1}{1 + \mathrm{e}^{-\beta}} > \int_{-\infty}^{2\beta/5} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t. \tag{7}$$
(i) If $11/4 \le \beta < 3$, we have $$\frac{1}{1 + \mathrm{e}^{-\beta}} - \int_{-\infty}^{2\beta/5} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t \ge \frac{1}{1 + \mathrm{e}^{-11/4}} - \int_{-\infty}^{2\cdot 3/5} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t > 0. $$
(ii) If $0 < \beta < 11/4$, it suffices to prove that $$\frac{1}{1 + \mathrm{e}^{-\beta}} > \frac{2\beta}{5\sqrt{2\pi}} + \frac12 > \int_{-\infty}^{2\beta/5} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t. \tag{8}$$
Let $F(\beta) := \frac{1}{1 + \mathrm{e}^{-\beta}} - \frac{2\beta}{5\sqrt{2\pi}} - \frac12$. We have $F''(\beta) := - \frac{\mathrm{e}^\beta(\mathrm{e}^\beta - 1)}{(1 + \mathrm{e}^\beta)^3} > 0$. Also, $F(0) = 0$ and $F(11/4) > 0 $. Thus, $F(\beta) > 0$ on $(0, 11/4)$.
Let $G(\beta) := \frac{2\beta}{5\sqrt{2\pi}} + \frac12 - \int_{-\infty}^{2\beta/5} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t$. We have $G'(\beta) = \frac{2(1 - \mathrm{e}^{-2\beta^2/25})}{5\sqrt{2\pi}} > 0$. Also, $G(0) = 0$. Thus, $G(\beta) > 0$ on $(0, 11/4)$.
Thus, (8) is true.
We are done.