The notion of the gradient in a multivariate function

Question

It is been widely said that the gradient of a multivariate function at a certain point shows the direction with which maximum increase can be attained. However, from another perspective, this claim seems wrong. Imagine the gradient of $f(x, y)$ at $(x_0, y_0)$ is $[7, 3]^T$ which indicates that any arbitrary small step $\varepsilon$ in the direction of $x$ gives $7/3$ times more progress than moving in the direction of $y$. Hence, based on intuition, one can claim that it is better to move only in the direction of $x$ which yields more progress. More precisely, if you just move on the $x$ axis by step size $\varepsilon$, you get $7 \varepsilon$ while obeying the gradient gives $(7\varepsilon/10)\times 7 +(3 \varepsilon /10) \times 3 = 5.8 \varepsilon $. And $5.8 \varepsilon < 7 \varepsilon $. Therefore, saying that "maximum increase can be attained if you follow the gradient" is wrong.

Answer 1

You have an error in how you've constructed the $\varepsilon$-length vector along the gradient, which leads you to an incorrect conclusion.

The length of the vector you've chosen is too short. By the pythagorean formula, it comes to $l = \varepsilon\cdot\sqrt{\frac{49}{100} + \frac{9}{100}} \approx 0.76\varepsilon$

The vector $[\frac{7}{\sqrt{58}} \varepsilon, \frac{3}{\sqrt{58}} \varepsilon]^T$ points in the direction of the gradient and has length $\varepsilon$. Moving along this vector would give an increase of $\sqrt{58}\varepsilon \approx 7.6\varepsilon$. As expected, this is a greater increase than you would get by moving just along the $x$-axis.