Interesting Gradient problem? Don't know how to write it?

Question

if $p=q+\varepsilon v$, where $v=\dfrac{\nabla f(q)}{\|\nabla f(q)\|}$.

$f(q)=0$. $\nabla f(x) $ is not $0$ on the domain.

How to prove there exists $\varepsilon_1>0$ such that if $\varepsilon_1>\varepsilon>0$, then $f(p)>0$?

I know gradient direction is the fastest increasing direction, so by the condition given, it seems obvious.

How to write it rigorously?

Answer 1

How to write this may depend on what has already been established. Suppose we can show that $$ \left.\frac d {d\varepsilon} f(p) \right|_{\varepsilon=0} >0. \tag 1 $$ Then the problem reduces to this: If $g(0)=0$ and $g'(0)>0$, how do we show that there is some $\varepsilon_1>0$ such that for all $\varepsilon$ between $0$ and $\varepsilon_1$ we have $g(\varepsilon)>0$?

That $g'(0)>0$ means that $$ D=\lim_{\varepsilon\to0}\frac{g(\varepsilon)}{\varepsilon} > 0. $$ So consider the interval $\left(\dfrac D 2,\dfrac{3D} 2\right)$. Certainly there exists $\varepsilon_1>0$ such that for all $\varepsilon$ within a distance of $\varepsilon_1$ from $0$, the fraction $g(\varepsilon)/\varepsilon$ is in the interval $\left(\dfrac D 2,\dfrac{3D} 2\right)$. If $\varepsilon$ is positive, that certainly means $g(\varepsilon)>0$ and so we're done.

The question is then how to prove $(1)$. And there I would try looking at the sort of chain rule that says $$ \frac d {d\varepsilon} f(p) = \nabla f(p)\cdot \frac{dp}{d\varepsilon} \text{ (a dot-product of two vectors)}. $$

Answer 2

I found this to be an engaging problem since it admits a somewhat more geometrical approach in the event that $\nabla f$ is assumed continuous, though as Michael Hardy has shown, a analytic technique suffices even when such an assumption is not made. And in response to his closing question, I would like to point out that the chain rule indeed applies and shows that $df(p)/d\varepsilon \mid_{\varepsilon = 0} > 0$ (this is verified in my proof below); for more on the chain rule the present context, see this wikipedia page.

In the event that $\nabla f$ is continuous near $q$, we can argue as follows:

Let $\gamma(s)$ be any differentiable path joining $q = \gamma(0)$ and $p = \gamma(\varepsilon)$; by the chain rule

$\dfrac{df(\gamma(s))}{ds} = \nabla f(\gamma(s)) \cdot \gamma'(s), \tag{1}$

and then since $f(q) = 0$,

$f(p) = f(p) - f(q) = \int_0^\varepsilon \dfrac{df(\gamma(s))}{ds} ds = \int_0^\varepsilon \nabla f(\gamma(s)) \cdot \gamma'(s) ds. \tag{2}$

We take the path $\gamma(s) = q + sv$ in (1); then $\gamma'(s) = v$ for all $s$ so that (2) becomes

$f(p) = \int_0^\varepsilon \nabla f(q + sv) \cdot v \; ds. \tag{3}$

Since

$v = \dfrac{\nabla f(q)}{\Vert \nabla f(q) \Vert}, \tag{4}$

the integrand in (3) becomes

$\theta(s) = \dfrac{\nabla f(q + sv) \cdot \nabla f(q)}{\Vert \nabla f(q) \Vert}; \tag{5}$

furthermore we have

$\theta(0) = \dfrac{\nabla f(q) \cdot \nabla f(q)}{\Vert \nabla f(q) \Vert} = \dfrac{\Vert \nabla f(q) \Vert^2}{\Vert \nabla f(q) \Vert} = \Vert \nabla f(q) \Vert > 0. \tag{6}$

$\theta(s)$ is a continuous function of $s$ since $\nabla f(q + sv)$ is continuous in $s$; it follows that for $m$ with $0 < m < \Vert \nabla f(q) \Vert$ there exists an $\varepsilon_1 > 0$ such that $\theta(s) > m$ if $s < \varepsilon_1$. Then if $0 < \varepsilon < \varepsilon_1$,

$f(p) = \int_0^\epsilon \theta(s) ds > \int _0^\varepsilon m ds = m \varepsilon > 0. \tag{7}$

This shows that $f(p) = f(q + \varepsilon v) > 0$ for $0 < \varepsilon < \varepsilon_1$, as per request. QED.

Hope this helps. Cheers,

and as ever,

Fiat Lux!!!