Little-o meaning in equation

352 Views Asked by At

I am completely new to little-o notation, I came across it in a lecture about algorithmically approaching a function minimum: $$ f(x^{n+1}) = f(x^n) + \nabla f(x^n ) \cdot (x^{n+1} − x^n) + o( \| x^{n+1} − x^n \| ) = f(x^n) + t \nabla f(x^n ) \cdot d^n + o(t) $$ where t is a chosen length (larger $t$ makes for larger steps, but possibly less direct).

To try to understand this, I searched and found this simple explanation: $$ f(x) = o(g(x)), \text{as } x\to x_0 $$ is equivalent to $\lim_{x\to x_0} f(x)/g(x) = 0$; pdf link

This explanation seems clear in itself, but I'm struggling to link it to the equation from my lecture notes. Can I rearrange the equation like this? $$ f(x^{n+1}) - f(x^n) - t \nabla f(x^n ) \cdot d^n = o(t) $$

Implying that: lim (of what variable as it goes to what value??) $[f(x^{n+1} ) - f(x^n ) - t \nabla f(x^n ) \cdot d^n]/t = 0$

There seems to be no indication of what variable/limit the o() is referring to in the notes. Should this be obvious?


Edit: Despite the answer below, I still have no idea how to interpret the above equation (and similar ones) from my lecture notes. From my perspective there seems to be a lack of information. If anyone has other answers please post them and I'll flag whichever answer makes this clearer for me.

2

There are 2 best solutions below

0
On BEST ANSWER

So you have:

$$f(x^{n+1})=f(x^n)+t \nabla f(x^n) \cdot d^n + o(t).$$

This means exactly that in some implicitly defined limit, $\frac{f(x^{n+1})-f(x^n)-t\nabla f(x^n) \cdot d^n}{t} \to 0$. The relevant limit here is as $t \to 0$. There isn't really a general rule to determine that, so when things might not be clear, it should be specified explicitly, rather than left to context. But here it is clear because this relationship is just giving the error estimate for the linear approximation of $f$ near $x^n$, going in the direction $d^n$.

2
On

It means that in the expression

$$f(x^{n+1}) = f(x^n) + t \nabla f(x^n ) \cdot d^n + o(t)$$

$o(t)$ indicates something going to zero faster than $t$, that is

$$o(t)=t\cdot\omega(t)$$

with $\omega(t) \to 0$ and therefore

$$\lim_{t\to 0} \frac{f(x^{n+1} ) - f(x^n ) - t \nabla f(x^n ) \cdot d^n}{t} = \lim_{t\to 0} \,\omega(t)=0$$