Let $f: \mathbb{R}^n \to \mathbb{R}$. Let $M$ be positive definite. Is it always true that $f(x)$ increases in the $M \nabla f$ direction, where $\nabla f$ is the direction of steepest ascent (the gradient)?
My intuition is yes, and I believe you can show it ''inductively" on coordinates using continuity. Is there an easier way?
On
Let scalar field $f : \Bbb R^n \to \Bbb R$ be differentiable. Let $n \times n$ matrix $\rm M$ be symmetric and positive definite. Choosing $\rm x \in\Bbb R^n$ such that $\nabla f ({\rm x}) \neq \Bbb 0_n$, the directional derivative of $f$ in the direction of vector ${\rm v} := {\rm M} \nabla f ({\rm x}) \neq \Bbb 0_n$ at $\rm x$ is
$$\lim_{h \to 0} \frac{f ({\rm x} + h {\rm v}) - f ({\rm x})}{h} = \langle \nabla f ({\rm x}), {\rm v} \rangle = \underbrace{\langle \nabla f ({\rm x}), {\rm M} \nabla f ({\rm x}) \rangle}_{> 0} > 0$$
which follows from the positive definiteness of matrix $\rm M$.
Yes, $f$ is locally increasing at $x$ in the direction $M\nabla f(x)$ for a positive definite matrix $M$. To see this, suppose $\nabla f(x) \ne 0$ and define $y = x + \alpha M\nabla f(x)$, where $\alpha > 0$. Then the Taylor series expansion of $f$ about $x$ gives \begin{equation*} f(y) = f(x) + \nabla f(x)^\top (y-x) + \text{h.o.t.}, \end{equation*} where h.o.t. denotes higher order terms in $y$. Substituting our expression for $y$, we find that \begin{equation*} f(y) - f(x) = \alpha \nabla f(x)^\top M \nabla f(x) + \text{h.o.t.}. \end{equation*} For sufficiently small $\alpha$, the higher order terms are dominated by the linear term of the Taylor series. Therefore, since $\alpha \nabla f(x)^\top M \nabla f(x) > 0$ by the assumptions that $\alpha>0$, $M\succ 0$, and $\nabla f(x)\ne 0$, we conclude that \begin{equation*} f(y) > f(x), \end{equation*} showing that moving away from $x$ in the $M\nabla f(x)$ direction results in a local increase in $f$.