The Theorem says:
Let $g,g'$ be continuous functions in the neighbourhood of an isolated fixed point $x_*=g(x_*)$. If $|g'(x_*)|<1$ then there is an interval $L=[x_*-\delta,x_*+\delta]$ such that $x_{k+1}=g(x_k)$ convergces to $x_*$ whenever $x_0\in L$.
The proof goes:
By continuity of $g'$ there exists some interval $L=[x_*-\delta,x_*+\delta]$ with $\delta>0$ such that $|g'(x)|\leq M$ for $M<1^{(1)}$. Not let $x\in L$. It follows that: $$ |x_*-g(x)|=|g(x_*)-g(x)|\leq^{(2)}M|x_* - x|<|x_* -x|\leq \delta $$
so $g(x)\in L$. Hence g is a contraction mapping on $L$ and the contracting mapping theorem (Banach fixed point theorem) shows that $x_k \rightarrow x$.
Now I have two questions about this proof:
(1) Why does this sentence hold? I know the theorem of Maxima and Minima for continuous functions and that they are bounded on a closed intervall but isn't this the other way around when we define a boundary and then say that a continuous function stays in this boundary on a certain (small enough) intervall? I cannot remember a theorem about that.
(2) This is the part I don't understand at all, why does this inequality hold? I guess it is a combination of $|g'(x)|\leq M$ and $|g'(x_*)|<1$, but how does one show this inequality formaly?
Appreciate your help and stay safe!
$(1)$:
$g'$ is continuous at $x_*$, then $\forall \epsilon > 0, \exists \hat{\delta} >0, |x-x_*|< \hat{\delta} \implies |g'(x)-g'(x_*)| < \epsilon$.
Note that if $|g'(x)-g'(x_*)|<\epsilon$ then we have $|g'(x)|<|g'(x_*)|+\epsilon$
Let $\epsilon = \frac{1-|g'(x_*)|}2>0$, then we have $|g'(x)|<|g'(x_*)|+\frac{1-|g'(x_*)|}{2} =\frac{1+|g'(x_*)|}2<1$.
Hence by the definition of continuity of $g'$ at $x_*$, we found an interval
$\forall x \in (x_*-\hat{\delta}, x_*+\hat{\delta}),|g'(x)|<\frac{1+|g'(x_*)|}2$
We can let $\delta = \frac{\hat{\delta}}2$ and $M=\frac{1+|g'(x_*)|}2$, then we have found an interval $L=[x_*-\delta, x_*+\delta]$ such that $|g'(x)|\le M<1$.
$(2)$:
We can use the mean value theorem, for any $x \in L$, we have
$$|g(x_*)-g(x)|=|g'(\zeta_x)||x_*-x|$$
where $\zeta_x$ lies between $x_*$ and $x$, hence $\zeta_x \in L$. Hence from part $(1)$, we know that $|g'(\zeta_x)|\le M$
$$|g(x_*)-g(x)|=|g'(\zeta_x)||x_*-x| \le M |x_*-x|$$