I am currently learning about different optimization methods. That is, I am interested in finding the maxima/minima of functions. The last method I have studied is Newton Raphson.
I understand Newton Raphson. As far as I understand it, Newtons method is simply the first order case of the Newton Raphson method. Is this correct?
Now, how do both of these methods relate to the concept of Taylor Series? It might be a trivial question but I am a bit confused, because these concepts seem to overlap a bit.
To my knowledge there is no distinction between Newton's method and the Newton-Raphson method, they are the same thing.
As for how Newton's method relates to power series. In the optimization context, Newton's method (iteratively) optimizes the best quadratic approximation to a function $f$ at a point $x_0$, which is given by truncating its power series at the second order term,
$$f(x)=\sum_{n=0}^\infty\frac{f^{(n)}(x_0)(x-x_0)^n}{n!}\approx f(x_0)+f'(x_0)(x-x_0)+\frac{1}{2}f''(x_0)(x-x_0)^2.$$
Taking the derivative of the quadratic approximation and setting it equal to zero gives us,
$$x=x_0-\frac{f'(x_0)}{f''(x_0)}.$$
Thus optimizing the best quadratic approximation to $f$ gives rise to the familiar root finding algorithm on $f'$, where each iteration in the root finding algorithm is equivalent to reforming the best quadratic approximation at each step of the optimization algorithm.