Newtons Method with an unknown method

Question

I am attempting to use Newtons method in an optimisation problem. I haven't used Newtons method since my college days and after refreshing my memory I am encountering the following challenge. Basically I have observation data point and a model to simulate the same point. There is a single coefficient I can tune to make the model more similar to the observation. Rather than tune it manually, I want to use something more efficient. I was thus recommended Newtons method. So using newtons method I make an initial guess of the coefficient and get an initial answer, I compare this against the observed result and get my initial error. So the next step is to find the tangent to the curve at that point, in order to find my second estimate. With the function being unknown it seems impossible to find the tangent? What should I do? The only solution that comes to mind is instead of making 1 single intitial guess, I have to make 3 guess’s and fit a function to them, and as I proceed, continue fitting functions to more and more points. is this the correct manner?

Answer 1

You can approximate the tangent's gradient at the point $x$ with parameter $p$ using a finite difference approximation:

$f'(x,p) \approx \frac{f(x,p+h)-f(x,p)}{h}$.

This will involve evaluating your unknown function $f$ at two points. You can choose $h$ to be a reasonably small number. If it is small enough, the approximation will work.

These evaluations will take a long time to do. Is there are a way you do two or more evaluations in parallel?