1-d case $\to$
GIVEN: Consider a line $f(x)$, Gradient- represents the change in $f(x)$ when you move by $h$, where $h$ is tending to zero.
Now, if I move in the direction of the gradient it presents the max. positive change in $f(x)$ which can be achieved by moving in any direction at that point. (Here, there are only 2 directions.)
What I don’t get? $h$ tends to zero. But, from what from what direction? It could be an infinitesimal small amount in left or right.
In $1$D the gradient is just the derivative. If it is not $= 0$ you are not in a stationary position, so you can climb, but you have no choice because you have only one direction available, and this is going to be automatically the steepest. If you were in more dimensions you'd have to find the steepest path, like being on the side of a mountain there is only one direction pointing directly to the top, but many climbind directions.