I have an infinitely differentiable function:
- $f(x)=\exp(k\ \cos(x))$
- $f^\prime(x) = -k\ \sin(x)\ \exp(k\ \cos(x))$
- $f^{\prime\prime}(x) = k\ \exp(k\ \cos(x))\ (k\ \sin^2(x) - \cos(x))$
- $f^{\prime\prime\prime}(x) = -k\ \exp(k\ \cos(x))\ \sin(x)\ (k^2\ \sin^2(x) -3k\ \cos(x) - 1)$
- etc...
Now $g$ is the sum of a few of these functions, shifted by various values:
- $g(x) = \sum_{n=1}^{N} f(x-\mu_n)$
- $\therefore g^{\prime}(x) = \sum_{n=1}^{N} f^{\prime}(x-\mu_n)$
- etc...
I want to find the local minimum of $-g(x)$ (or local maximum of $g(x)$) closest to a given starting point, $x_0$.
I tried using the Newton-Rhapson method (specifically the Scipy implementation) to find the closest root of $g^{\prime}(x)$ to $x_0$. This method seems to be significantly faster than the other functions in that library, which do not use the derivatives at all. However, it sometimes gives me a maximum instead of a minimum, since I can't guarantee that $x_0$ is closer to the nearest local maximum of $g(x)$ than it is to the nearest local minimum of $g(x)$.
Since the derivatives are relatively cheap to calculate, which other minimization functions use the derivatives to speed up the convergence?
Alternatively, is there a modified version of the Newton-Rhapson method that takes the sign of $g^{\prime\prime}$ into account, so that it only finds roots of $g^\prime$ where $g^{\prime\prime} < 0$? Such roots would correspond to maxima in $g$, which solves my original problem.