Analyzing the convergence/iteration complexity of an algorithm with geometric convergence

77 Views Asked by At

Consider an algorithm with a convergence rate given by: O((1-1/\kappa)^t). How does one show that this gives an iteration complexity of O(\kappa*1/\epsilon)?

Setting O((1-1/\kappa)^t)=\epsilon and solving for t just seems to give that t=O(log\epsilon).

1

There are 1 best solutions below

0
On BEST ANSWER

Ignoring the $O(\cdot)$, which doesn't change the argument.

You cannot get an interation complexity with an inverse linear dependence on $\varepsilon:$ that rate gives a logarithmic dependence.

Specifically, you have a convergence rate of $$ f(t) = \left(1-\frac{1}{\kappa}\right)^t$$ so to achieve an error $\varepsilon$, you need at least $T$ iterations, where $f(T) \leq \varepsilon$, i.e., $$ \left(1-\frac{1}{\kappa}\right)^T \leq \varepsilon $$ which, taking the logarithm and reorganizing, leads to $$ T \geq \log\frac{1}{\varepsilon} \cdot \frac{1}{-\log(1-\kappa^{-1})} = \Theta\!\left(\kappa\log\frac{1}{\varepsilon}\right) $$ assuming $\kappa \gg 1$, along with $-\log(1-x) = \Theta(x)$ when $x\to 0$.