Consider the product $\left|w\sigma'(wa+b)\right|$, where $\sigma(x) = \frac{1}{1+e^{-x}}$. Suppose that $|w| \geq 4$, and $|w\sigma'(wa+b)|\geq 1$. Show that the set of $a$ satisfying that constraint can range over an interval no greater in width than
$$\frac{2}{|w|}\ln\Bigg( \frac{|w|(1+\sqrt{1−4/|w|}}{2}−1\Bigg).$$
Not quite sure where to start, I tried to work with the original equation:
$$e^{wa+b}/\left(e^{wa+b}+1\right)^2 \geq 1/|w| \implies wa+b - 2\ln\left(e^{wa+b}+1\right) \geq -\ln(w).$$
However, this doesn't seem to be going anywhere, and I'm just not sure how to go about finding the width of an $\textit{interval}$. Appreciate any suggestions/solutions, thanks!
Starting with
$$\lvert w\sigma'(wa+b)\rvert \geq 1$$
Using the definition of the sigmoid and its derivative,
$$\lvert w\rvert \frac{e^{wa+b}}{(1+e^{wa+b})^2} \geq 1$$
Let $u \equiv e^{wa+b}$. Then,
$$\lvert w\rvert \frac{u}{(1+u)^2} \geq 1$$
Rearranging,
$$u^2+(2-\lvert w\rvert)u+1 \leq 0$$
Solving for $u$ gives
$$\frac{(\lvert w\rvert-2)-\sqrt{w^2-4\lvert w\rvert}}{2} \leq u \leq \frac{(\lvert w\rvert-2)+\sqrt{w^2-4\lvert w\rvert}}{2}$$
This is where the interval comes from. Before we get the width of that interval, we must finish solving for $a$. Note that we know both interval bounds are real-valued because $\lvert w \rvert \geq 4$. Plugging in for the definition of $u$ and doing some rearranging to better match the answer,
$$\frac{\lvert w\rvert\left(1-\sqrt{1-4/\lvert w\rvert}\right)}{2}-1 \leq e^{wa+b} \leq \frac{\lvert w\rvert\left(1+\sqrt{1-4/\lvert w\rvert}\right)}{2}-1$$
From here you can quickly solve for $a$:
$$\frac{1}{\lvert w\rvert}\ln\left(\frac{\lvert w\rvert\left(1-\sqrt{1-4/\lvert w\rvert}\right)}{2}-1\right) - \frac{b}{\lvert w\rvert}\leq a \leq \frac{1}{\lvert w\rvert}\ln\left(\frac{\lvert w\rvert\left(1+\sqrt{1-4/\lvert w\rvert}\right)}{2}-1\right)-\frac{b}{\lvert w\rvert}$$
So you can see $a$ lies on that interval. To get the width of the interval, simply subtract the lower bound from the upper bound:
$$\left[ \frac{1}{\lvert w\rvert}\ln\left(\frac{\lvert w\rvert\left(1+\sqrt{1-4/\lvert w\rvert}\right)}{2}-1\right)-\frac{b}{\lvert w\rvert}\right]-\left[\frac{1}{\lvert w\rvert}\ln\left(\frac{\lvert w\rvert\left(1-\sqrt{1-4/\lvert w\rvert}\right)}{2}-1\right) - \frac{b}{\lvert w\rvert}\right]$$
$$=\frac{1}{\lvert w\rvert}\ln{\left(\frac{\lvert w\rvert\left(1+\sqrt{1-4/\lvert w\rvert}\right)-2}{\lvert w\rvert\left(1-\sqrt{1-4/\lvert w\rvert}\right)-2}\right)}$$
Notice we lose the dependence on $b$. Multiplying the numerator and denominator inside the logarithm by $\lvert w\rvert\left(1+\sqrt{1-4/\lvert w\rvert}\right)-2$ and simplifying:
$$=\frac{1}{\lvert w\rvert}\ln{\left(\left[\frac{\lvert w\rvert\left(1+\sqrt{1-4/\lvert w\rvert}\right)}{2}-1\right]^2\right)}$$
$$=\frac{2}{\lvert w\rvert} \ln{\left(\frac{\lvert w\rvert(1+\sqrt{1-4/\lvert w\rvert})}{2}-1\right)}$$
This is the width of the interval. From our solution to $a$, you can see that the interval is centered about $-b/\lvert w\rvert$.