Consider a function: $f: \mathbf{R} \to \mathbf{R}$ that has $f^\prime(x) < 0$ for some $x$. It is easy to show using taylor approximation or even the definition of derivative to show that for small enough $h$, we have $f(x + h) < f(x)$.
I am struggling to extend this to higher dimensions in rigorous way. Consider, $f: \mathbf{R}^n \to \mathbf{R}$ that has a gradient at $x$ statisfying: $\nabla f(x)^Th < 0$. How do i show rigorously that for some $\epsilon$ small enough we have $f(x + \epsilon h) < f(x)$.
I manage to prove this by defining a function $g$ in one variable on the path between $x$ and $x + h$ and using taylor expansion in single variable.
But alternatively, when I try to use the definition of derivative:
$f(x + h) = f(x) + \nabla f(x)^Th + o(||h||)$,
where $o(||h||)$ is the little-o notation, meaning that $\frac{o(||h||)}{||h||} \to 0$ as $||h|| \to 0$
I am stuck mainly because I don't know what to do with the $o(||h||)$ term. Can someone help me with this particular approach for a rigorous proof.
The differentiability of $f$ at $x$ means for any $\epsilon > 0$ there exists $\delta(\epsilon) > 0$ such that if $\|h \| < \delta(\epsilon)$ we have
$$|f(x + h) - f(x) - \nabla f(x)^T \cdot h| < \epsilon \|h\|.$$
Since, $f(x + h) - f(x) - \nabla f(x)^T \cdot h\leqslant |f(x + h) - f(x) - \nabla f(x)^T \cdot h| $, this implies
$$\tag{*}f(x+h) - f(x) < \nabla f(x)^T \cdot h + \epsilon \|h\|$$
If $\nabla f(x)^T \cdot h < 0$ as you suppose we can choose $\epsilon_0$ so small that the RHS of (*) is also less than $0$ when $\|h\| < \delta(\epsilon_0)$.