Proof for multivariate Newton-Raphson method

Question

How can the proof for Newton's method for a single variable be extended to the multivariate version?

Forgive me if this is trivial, but I don't seem to get it. Any links or proofs would be greatly appreciated!

Answer 1

Let's say $x_0$ is a root. We must assume that the Jacobian matrix $Df(x_0)$ is invertible; otherwise the quadratic convergence may fail (as it does in the single-variable case, when $f'(x_0)=0$). Also assume that $f$ is $C^2$ smooth (consequently, $Df$ is Lipschitz continuous). Then at $x=x_0+h$, with $h$ small, we have $$f(x) = Df(x_0) h + O(\|h\|^2)\tag1$$ (second order Taylor expansion) and $$Df(x) = Df(x_0) + O(\|h\|)\tag2$$ (Lipschitz continuity of $Df$). Since $Df(x_0)$ is invertible, (2) implies $$Df(x)^{-1} = Df(x_0)^{-1} + O(\|h\|)\tag3$$ From (1) and (3), $$\begin{split}x-(Df(x))^{-1}f(x) &= x_0+h - (Df(x_0)^{-1} + O(\|h\|)) (Df(x_0) h + O(\|h\|^2))\\ & = x_0 +O(\|h\|^2) \end{split}$$ which is the desired quadratic convergence.