I used to Numerical Analysis by Richard L. Burden for my $3$rd year numerical analysis course. But the section $10$ (Numerical solutions of nonlinear systems of equations) wasn't cover rigorous proof or analysis for $(i)$ Fixed points for functions of several variables, $(ii)$ Newton's Method, $(iii)$ Steepest descent Techniques.
Like they state Theorem $10.6$,
Let $D=\left\{\left(x_{1}, x_{2}, \ldots, x_{n}\right)^{t} \mid a_{i} \leq x_{i} \leq b_{i}\right.$, for each $\left.i=1,2, \ldots, n\right\}$ for some collection of constants $a_{1}, a_{2}, \ldots, a_{n}$ and $b_{1}, b_{2}, \ldots, b_{n} .$ Suppose $\mathbf{G}$ is a continuous function from $D \subset \mathbb{R}^{n}$ into $\mathbb{R}^{n}$ with the property that $\mathbf{G}(\mathbf{x}) \in D$ whenever $\mathbf{x} \in D$. Then $\mathbf{G}$ has a fixed point in $D$. Moreover, suppose that all the component functions of $\mathbf{G}$ have continuous partial derivatives and a constant $K<1$ exists with $$ \left|\frac{\partial g_{i}(\mathbf{x})}{\partial x_{j}}\right| \leq \frac{K}{n}, \quad \text { whenever } \mathbf{x} \in D $$ for each $j=1,2, \ldots, n$ and each component function $g_{i}$. Then the sequence $\left\{\mathbf{x}^{(k)}\right\}_{k=0}^{\infty}$ defined by an arbitrarily selected $\mathbf{x}^{(0)}$ in $D$ and generated by $$ \mathbf{x}^{(k)}=G\left(\mathbf{x}^{(k-1)}\right), \quad \text { for each } k \geq 1 $$ converges to the unique fixed point $\mathbf{p} \in D$ and $$ \left\|\mathbf{x}^{(k)}-\mathbf{p}\right\|_{\infty} \leq \frac{K^{k}}{1-K}\left\|\mathbf{x}^{(1)}-\mathbf{x}^{(0)}\right\|_{\infty} $$
But didn't give any rigorous proof of that. I suspect Contraction mapping from Lipschitz continuity could be used here in order to proof it.
- Like for single equation we look for such rearrangement of $g(x)=x$ such that $|g(x)|<1$ near the original root. But how to interpret same thing here?
Like Newton's method for single equation $g(x)=x-(1/f'(x))f(x)$, here they requires that $A(x)$ so that, $$G(x)=x-A(x)^{-1}F(x),\quad \text{where }A(\mathbf{x})=\left[\begin{array}{cccc} a_{11}(\mathbf{x}) & a_{12}(\mathbf{x}) & \cdots & a_{1 n}(\mathbf{x}) \\ a_{21}(\mathbf{x}) & a_{22}(\mathbf{x}) & \cdots & a_{2 n}(\mathbf{x}) \\ \vdots & \vdots & & \vdots \\ a_{n 1}(\mathbf{x}) & a_{n 2}(\mathbf{x}) & \cdots & a_{n n}(\mathbf{x}) \end{array}\right]$$
And state Theorem $10.7$,
Let $\mathbf{p}$ be a solution of $\mathbf{G}(\mathbf{x})=\mathbf{x}$. Suppose a number $\delta>0$ exists with
(i) $\partial g_{i} / \partial x_{j}$ is continuous on $N_{\delta}=\{\mathbf{x} \mid\|\mathbf{x}-\mathbf{p}\|<\delta\}$, for each $i=1,2, \ldots, n$ and $j=1,2, \ldots, n$
(ii) $\partial^{2} g_{i}(\mathbf{x}) /\left(\partial x_{j} \partial x_{k}\right)$ is continuous, and $\left|\partial^{2} g_{i}(\mathbf{x}) /\left(\partial x_{j} \partial x_{k}\right)\right| \leq M$ for some constant $M$, whenever $\mathbf{x} \in N_{\delta}$, for each $i=1,2, \ldots, n, j=1,2, \ldots, n$, and $k=1,2, \ldots, n$;
(iii) $\quad \partial g_{i}(\mathbf{p}) / \partial x_{k}=0$, for each $i=1,2, \ldots, n$ and $k=1,2, \ldots, n$.
And define,
$$ A(\mathbf{x})=J(\mathbf{x}),\quad\text{where } J(\mathbf{x})=\left[\begin{array}{cccc} \frac{\partial f_{1}}{\partial x_{1}}(\mathbf{x}) & \frac{\partial f_{1}}{\partial x_{2}}(\mathbf{x}) & \cdots & \frac{\partial f_{1}}{\partial x_{n}}(\mathbf{x}) \\ \frac{\partial f_{2}}{\partial x_{1}}(\mathbf{x}) & \frac{\partial f_{2}}{\partial x_{2}}(\mathbf{x}) & \cdots & \frac{\partial f_{2}}{\partial x_{n}}(\mathbf{x}) \\ \vdots & \vdots & & \vdots \\ \frac{\partial f_{n}}{\partial x_{1}}(\mathbf{x}) & \frac{\partial f_{n}}{\partial x_{2}}(\mathbf{x}) & \cdots & \frac{\partial f_{n}}{\partial x_{n}}(\mathbf{x}) \end{array}\right] $$
I didn't understand how they justify $A(x)=J(x)$ here?
Any suggestion of rigorous textbooks which cover the topics I mentioned above or lecture notes with proofs are appreciated.