It's not an exercise for uni or anything like that, just something that's been bothering me a bit and I can't seem to find useful information on the web on the matter.
When talking about real valued scalar functions, we know that newton's method will surely converge to a root $s$ of $f(x)$ if our initial value $x_0$ is sufficiently close to the root, meaning, on the interval $(s-r,s+r)$ where $r=|s-x_0|$ the derivative $f'(x)$ is never zero (assuming $f'(s)$ is not zero).
My question is, can we expand that criterion to higher dimension?
Let $f: \mathbb R^{n} \to \mathbb R^n$ differentiable function with continuous partial derivatives. Let $s\in \mathbb R^n$ such that $f(s)=0$ and let $x_0 \in \mathbb R^n$ and $r=|s-x_0|$.
Assume the jacobian of $f$ is invertible everywhere in the sphere with radius $r$ at epicenter $s$. Prove or disprove that newton's method will converge to $s$ if our initial value was $x_0$.
Reminder:
Newton's method is defined as $x_{n+1}=x_n-J^{-1}(x_n)f(x_n)$
This generalization of Newton's method works as well.
Let $\varepsilon > 0$ to be chosen later. Let $\delta$ be such that $\|J(x)-J(x_0)\| \leq \varepsilon$ and $\|J^{-1}(x)-J^{-1}(x_0)\| \leq \varepsilon$ on a $\delta$-neighborhood of $x_0$.
For $x \in \overline{B}(x_0, \delta)$, let $T(x) := x-J^{-1}(x)f(x)$. This is well-defined if $\varepsilon < \|J^{-1}(x_0)\|$. Then :
$$f(x) = \int_0^1 J(x_0+t(x-x_0)) \cdot (x-x_0) \ dt.$$
Let me write $J_t (x) := J(x_0+t(x-x_0))$. Then :
$$J^{-1}(x)f(x) = \int_0^1 J^{-1}(x) \cdot J_t(x) \cdot (x-x_0) \ dt = x-x_0 + \int_0^1 [J^{-1}(x) \cdot J_t(x) - I]\cdot (x-x_0) \ dt$$
$$\|T(x)-x_0\| = \left\| \int_0^1 [J^{-1}(x) \cdot J_t(x) - I]\cdot (x-x_0) \ dt \right\| \leq \left( \max_{t \in [0,1]} \left\| J^{-1}(x) \cdot J_t(x) - I \right\| \right) \cdot \|x-x_0\|$$
But, for all $x$ and $y$ in the closed $\delta$-neighborhood of $x_0$, we know that $\|J(x)-J(y)\| \leq 2 \varepsilon$. Hence,
$$\max_{t \in [0,1]} \left\| J^{-1}(x) \cdot J_t(x) - I \right\| \leq \|J^{-1}(x)\| \cdot \left( \max_{t \in [0,1]} \left\| J_t(x) - J(x) \right\| \right) \leq 2 \varepsilon \left(\|J^{-1}(x_0)\|+\varepsilon \right).$$
Now, choose $\varepsilon < \|J^{-1}(x_0)\|$ such that the right hand side is, say, at most $1/2$. Note that it implies that $T$ maps $\overline{B}(x_0, \delta)$ into itself, so we can iterate $T$ as many times as we want. Then for all $x$ in $\overline{B}(x_0, \delta)$, for all $n \geq 0$:
$$\|T^n (x)-x_0 \| \leq \delta 2^{-n},$$
so Newton's method converges at least at (actually faster than) exponential speed.