Suppose that $f: \mathbb R^n \to \mathbb R$ is differentiable. I would like to verify the following claim:
Given a vector $p \in \mathbb R^n$ such that $p^T \nabla f(x) < 0 $ there exists an $\alpha \ge 0 $ such that
$$f(x + \alpha p) < f(x)$$
I have seen motivations pointing to Taylor's theorem, but all I can understand is the following:
From Taylor's theorem [for multivariate real-valued functions]:
$$f(x + \epsilon p) = f(x) + \epsilon p^T \nabla f(x) + O(\epsilon ^2)$$
Where of course, for $\epsilon > 0$
$$f(x) + \epsilon p^T \nabla f(x) < f(x) $$
But how do we make sure that the error term $O(\epsilon ^2) $ doesn't make up for the difference bwteeen $f(x + \epsilon p)$ and $f(x) + \epsilon p^T \nabla f(x)$? Or differently put, how do we make sure that
$$ \epsilon p^T \nabla f(x) + O(\epsilon ^2) < 0$$ $$?$$
This is an attempt to a solution following the comment from Thorgott.
Write $h(e) $ for the error function in the Taylor expansion that is $O(\epsilon ^2) $ and pick M and delta such that for $0<\epsilon < \delta$ we have that
$$h(\epsilon) \le M \epsilon ^2$$
Then consider
$$\epsilon p^T \nabla f(x) + h(\epsilon) \le \epsilon p^T \nabla f(x) + M \epsilon ^2 = \epsilon \left ( p^T \nabla f(x) + M \epsilon \right )$$
Since this expression holds for any $\epsilon < \delta $ we may decrease $\epsilon$ such that $M \epsilon < | p^T \nabla f(x) | $. Then certainly the expression inside the parenthesis is negative.