I will give an example to illustrate the question:
Assume I have the system:
$$ xy + x + y = 7\\ x^2 + y^3 = 9 $$
and I want to solve for $x$ and $y$. It is a fairly common approach to rearrange the first equation for $x$ and the second for $y$ as follows:
$$ x = \frac{7 - y}{1+y}\\ y = (9 - x^2)^{\frac{1}{3}} $$ then to choose a pair of starting values $x_0,y_0$ and iterate between these until (hopefully) the results converge on a solution.
For this example the solution does converge to $x = 1.91$ and $y = 1.75$ after about 15 iterations (starting with $x_0 = y_0 = 1$) and these values are indeed solutions.
My questions are:
1) When will this approach be successfull? Can we know a priori if it will be likely to work or not work on a given equation set.
2) Why does it work at all, is there any underlying theory which has been developed to understand the convergence of these schemes which iterate through a given equation set?
3) Does this approach and area have a specific name? (More specific than just 'solving systems of nonlinear equations').