We have $A=\iint {\rm dx}' {\rm d}y'=\iint G \,{\rm d}x\,{\rm d}y$, where the integral is over a region with area $A$ in the $xy$-plane and $G$ the Jacobian of the coordinate transformation $x\to x'$ and $y\to y'$, but isn't $A=\iint {\rm d}x\,{\rm d}y$ also, so $G=1$?
Why $A=\iint {\rm d}x\,{\rm d}y$ isnt true?
On
This is similar to the answer from anomaly, but I hope to make it easier to understand by moving the essential point out of a little subscript and into the text. The first equation in the question, $A=\iint dx'\,dy'$ is fine when the integral is over $A$, as you said. But the next bit, $\dots=\iint G\,dx\,dy$ requires that the integral be over a different region, which anomaly called $\Omega$. This is the region in the $x$-$y$-plane that corresponds, via the transformation $\hat G$ whose Jacobian is $G$, to the region $A$ in the $x'$-$y'$-plane. In other words, when $(x,y)$ and $(x',y')$ are related by $\hat G$, then the domain over which $(x,y)$ ranges is not usually the same as the domain over which $(x',y')$ ranges, and you need to take that into account when setting the domains of integration in your formula.
Edit: Rereading anomaly's answer, I see that (s)he used $\Omega$ to refer to two different regions, so you might do well to ignore the reference to that in my answer.
On
It's only in cartesian (rectangular) coordinates that $A = \iint_\Omega{\rm d}x\,{\rm d}y$ denotes the area of the region $\Omega$. When we change to a different coordinate-system then the Jacobian describes how the area of a small element of size ${\rm d}x'\,{\rm d}y'$ in the new coordinates is related to the corresponding area of size ${\rm d}x\,{\rm d}y$ in cartesian coordinates.
Consider for example the case of going from cartesian $(x,y)$ to polar coordinates $(r,\theta)$ (see the plot below). The area of a small element of extent ${\rm d}r$ and ${\rm d}\theta$ is seen to be ${\rm d}A = r\,{\rm d}r\,{\rm d}\theta$. The factor of $r$ is the Jacobian in this case.
$~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$
If the region in question is a circle of radius $R$ then $\Omega = \{(x,y): x^2+y^2 \leq R^2\}$ and
$$A = \iint_\Omega {\rm d}x\,{\rm d}y = \int_{-R}^R\int_{-\sqrt{R^2-x^2}}^{\sqrt{R^2-x^2}}{\rm d}y = \pi R^2$$
In polar coordinates on the other hand we have $\Omega = \{(r,\theta) : 0\leq r\leq R, \theta\in[0,2\pi] \}$ so
$$A = \iint_\Omega r\,{\rm d}r\,{\rm d}\theta = \int_0^R r\,{\rm d}r\int_0^{2\pi}{\rm d}\theta = \pi R^2$$
The inclusion of the Jacobian factor $r$ is crucial to get the right answer and if we try to compute it as $A = \iint_\Omega {\rm d}r\,{\rm d}\theta = \int_0^R{\rm d}r\int_0^{2\pi}{\rm d}\theta = \pi R$ we would get the wrong result.
The area $A$ of a region $\Omega$ is by definition $A = \int_{\Omega} dx\, dy$, where $x,y$ are the usual rectangular coordinates. (I'm trying to avoid discussing what exactly kind of object $dx\, dy$ is; more formally, we should talk about differential forms, Lebesgue integrals, etc. here.) The $x$ and $y$ above are not arbitrary; if you want to write $A$ in terms of some other coordinate system $z, w$, then the integral above becomes $\int_\Omega G(z, w) \, dz\, dw$ by the change of variables theorem.