Is there a way of being sure that simple iteration schemes, such as Gauss-Jacobi and Gauss-Seidel will converge for non-linear systems? I understand that for linear systems, the matrix A has to be diagonally dominant for G-J iteration, etc. to converge but can't seem to find anything online about convergence for non linear systems. An example may help:
$x(A-3x-4y)=0$ ;
$y(B-2x-y)=0$
Thanks, been struggling for a while.
We focus on the second example given by the OP $$x_{j+1}=1-y_j,\\y_{j+1}=\sqrt{9-x_{j+1}^2}.$$
We can eliminate $x$ to reduce to a single variable problem
$$y_{j+1}=\sqrt{9-(1-y_j)^2}=f(y_j).$$
By solving the equation, we known that
$$y=\frac{\sqrt{17}+1}2$$ is a fixed-point.
As the derivative
$$|f'(y)|=\left|\frac{1-y}{\sqrt{9-(1-y)^2}}\right|=\left|\frac{\sqrt{17}-9}8\right|<1$$ at the fixed point, linear convergence is guaranteed (we have a contracting map).
Anyway, we have to discuss the attraction basin of this fixed-point.
$f$ is defined over $[-2,4]$, has a single maximum at $(1,3)$ and range $[0,3]$. $f$ is also convex. So after one iteration, $f(y)\in[0,3]$, and after two iterations $f(y)\in[\sqrt5,\sqrt8]$, where the function is monotonous.
Then the largest slope in this range is at $\left(\sqrt8,\dfrac{1-2\sqrt2}{2\sqrt[4]2}\right)$, so that the Lipschitz constant $0.8$ holds. Convergence is guaranteed starting from any $y\in[-2,4]$.