The steepest descent direction at iteration $k$ is given by $$ \textbf{d}_k = -\textbf{g}_k = -\nabla f(\textbf{x}^{(k)}) $$ We now relax this requirement and impose the less constraining condition $$ \forall k \geq 0, \quad -\textbf{g}_k^\top \textbf{d}_k \geq \gamma \|\textbf{g}_k\|\|\textbf{d}_k\| \quad \text{where } 0<\gamma <1 \text{ does not depend on }k $$ In other words, the angle between $\textbf{d}_k$ and $-\textbf{g}_k$ must be acute. Furthermore, the cosine between these vectors must remain larger than $\gamma$ (uniformly in $k$)
My question: Why does the angle has to be acute?
Here is my attempt for explanation:
Recall: law of cosine: $$ \textbf{v} \cdot \textbf{w} = \|\textbf{v}\| \|\textbf{w}\| \cos \theta \iff \frac{\textbf{v} \cdot \textbf{w}}{\|\textbf{v}\| \|\textbf{w}\|} = \cos \theta $$ hence \begin{align*} -\textbf{g}_k^\top \textbf{d}_k &\geq \gamma \|\textbf{g}_k\|\|\textbf{d}_k\|\\ -\|\textbf{g}_k\|\|\textbf{d}_k\| \cos \theta &\geq \gamma \|\textbf{g}_k\|\|\textbf{d}_k\|\\ -\cos \theta &\geq \gamma \\ -\cos \theta &\geq 0 \\ \end{align*}
But it doesn't seem to help to find the answer
By a property of the cosine
$$ \cos \langle g,d \rangle = \frac{g\cdot d}{||g||\,||d||}$$
your equation says
$$\cos \langle g,d \rangle \leq -\gamma$$
Now this seems odd to me; to have an acute angle the logical requirement should be
$$\cos \langle g,d \rangle \geq \gamma$$
I suspect there is a mistake in the text you're given. The above relationship is important because, if your objective function is of class $C^1$, then a (negative, first-order) Taylor expansion (i.e., negative gradient) at each point gives you the direction of greatest descent, so you want to have your next point within a $90$ degree range of this direction, otherwise you will be ascending.