The Newton-Raphson method is known to be used in root-finding by linearizing the problem around the operating point and then inverting it, to give a better solution (closer to zero).
$$u_{i+1} = u_{i} - J^{-1}f_{(u)}$$
where $f_{(u)} = 0$ once the solution is reached, $J$ is Jacobian matrix, where the elements are partial derivatives of $f_{(u)}$ according to $u$.
Question:
Can we go futher than the first order linearization ? How about using second and third order derivatives, when solving root-finding problems (where $u$ and $f_{(u)}$ are vectors)?
Reason:
I am currently attaining to solve a problem, where one of the functions within the vector $f_{(u)}$ causes problems when the linear approximation is used. I see the Newton-Raphson's linearization hugely underestimates the particular function, causing the problem to converge at extremely slow pace.
More details about the problem:
The essential part of the problem is as follows:
$$f_{1(u_1)} = f_{2(u_2)} + f_{3(u_2)}$$ $$ u_1 = u_2 $$
where $f_{1(u_1)}$ is linear with derivative around 0.01, $f_{2(u_2)}$ is exponential and $f_{3(u_2)}$ is linear with derivative around 1000.
The system is very sensitive to change in variables Its the exponential function, when approximated linearly, the approximation is very bad. The system is very sensitive to change in variables $u_1$ and $u_2$. It takes around 5000 N-R iterations to converge. The N-R does not oscillate, just the update rate is extremely low.
Muller's method uses quadratic approximation (it constructs a parabola and marks its intersection with $x$ axis as an approximation of the next root), Halley's method uses second derivative to perform an update. In general case have a look at Householder's method: