Let $A=\{(x_1,x_2)\in\mathbb R^2| \max\{ |x_1|,|x_2|\}<1\}$ and $g(x_1,x_2)=\frac14(x_1^2x_2,x_1+1)$.
I want to show that $g$ has a fixed point in $A$.
So $|g(x)|\leq\max\{\frac14|x|^3,\frac14(|x|+1)\}\leq\frac12<1$ and so $g(A)\subset A$.
Also we have $|g(x)-g(y)|\leq \max\{\frac14|x_1^2x_2-x_1^2y_2|+\frac14|x_1^2y_2-y_1^2y_2|,\frac14|x_1-y_1|\}\leq \frac34|x-y|$ and therefore it's Lipschitz with $L=\frac34<1$.
So by the fixed point theorem of Banach we get a fixed point.
The only thing I've missed is the completeness of $A$.
So let $(a_n)$ be a Cauchy-sequence in $A$. Then it's a Cauchy-sequence in $\mathbb R^2$ and there converges to $a\in\mathbb R^2$. But why is $a\in A$?
$A$ isn't complete in the usual metric. I suggest you forget about fancy fixed-point theorems and just solve the simultaneous equations $x_1=\frac14x_1^2x_2$ and $x_2=\frac14(x_1+1)$ that define fixed points of $g$