Gradient descent proof: justify $\left(\dfrac{\kappa - 1}{\kappa + 1}\right)^2 \leq \exp(-\dfrac{4t}{\kappa+1})$

Question

A claim on pg 279 of the notes states that:

How can the encircled be justified? Note that $\kappa := \beta /\alpha$ is a non-negative constant.

I tried using the definition of the exponential:

$$e^z = \lim\limits_{n \to \infty} (1+z/n)^n$$ to no avail, because I cannot produce such an $n$ for the term $\left(\dfrac{\kappa - 1}{\kappa + 1}\right)^2$

Can someone please show how this inequality was attained?

Answer 1

You overlooked that in the last inequality the iterate changes to $1$ from $t$. So the inequality you need is just \begin{align*} \left(\left(1 - \frac {2} {\kappa + 1} \right)^2 \right)^t \le \left(e^{ \frac{-4} {\kappa +1} } \right)^t. \end{align*} For the remaining part you only need to note \begin{align*} \left(1 - \frac {2} {\kappa + 1} \right)^2 = 1 - \frac{4} {\kappa + 1} + \frac{4}{(\kappa +1 )^2} \\ e^{ \frac{-4} {\kappa +1} } = 1 - \frac{4}{\kappa + 1} + \frac{8}{(\kappa +1)^2} - \dots \ge 1 - \frac{4} {\kappa + 1} + \frac{4}{(\kappa +1 )^2}. \end{align*}