I'm trying to show that the area bound by the curves $r^2= 3\cos(2\theta)$, $r^2= 4\cos(2\theta)$, $r^2= 3\sin(2\theta)$, $r^2= 4\sin(2\theta)$ in the first quadrant is equal to $$A= \frac{10 - 7\sqrt{2}}{4} $$ using a change of variables. Most of my tries have ended with integrals that don't have a solution. The closest I got was using $$\cos(2\theta)=r^2/u $$ $$\sin(2\theta)=r^2/v $$ But unfortunately that hasn't worked either, because the indefinite integral does not have a solution. Any ideas?
Any suggestions are much appreciated!
I don't think you need substitution for this problem, and I think you are missing a factor of 2 possibly because you didn't consider quadrant III.
We start with $$ A=\iint \frac12\cdot 1_A\,\mathrm{d}(r^2)\,\mathrm{d}\theta $$ Clearly $2\theta$ is in the first quadrant, to get $\sin(2\theta)\geq 0$ and $\cos(2\theta)\geq 0$. So with a bit more thought, $$ A=\int_0^{\pi/4}\int_{r^2=3\max(\sin2\theta,\cos2\theta)}^{r^2=4\min(\sin2\theta,\cos2\theta)} 1_{3\max(\sin2\theta,\cos2\theta)\leq 4\min(\sin2\theta,\cos2\theta)}\,\mathrm{d}(r^2)\,\mathrm{d}\theta $$ since the quadrant III region is just the rotation of the quadrant I region about the origin.
Examining $0\leq3\max(\sin2\theta,\cos2\theta)\leq4\min(\sin2\theta,\cos2\theta)$ more closely gives $2\theta\in[\arctan(3/4),\arctan(4/3)]$, so \begin{align*} A&=\int_{2\theta=\arctan(3/4)}^{2\theta=\pi/4}\int_{r^2=3\cos2\theta}^{r^2=4\sin2\theta} \,\mathrm{d}(r^2)\,\mathrm{d}\theta+ \int_{2\theta=\pi/4}^{2\theta=\arctan(4/3)}\int_{r^2=3\sin2\theta}^{r^2=4\cos2\theta} \,\mathrm{d}(r^2)\,\mathrm{d}\theta\\ &=2\int_{2\theta=\arctan(3/4)}^{2\theta=\pi/4}(4\sin2\theta-3\cos2\theta)\,\mathrm{d}\theta\\ &=\left[-4\cos2\theta-3\sin2\theta\right]_{2\theta=\arctan(3/4)}^{2\theta=\pi/4}\\ &=\left[-4\cdot\frac1{\sqrt2}-3\cdot\frac1{\sqrt2}\right]-\left[-4\cdot\frac45-3\cdot\frac35\right]\\ &=\frac{10-7\sqrt2}2 \end{align*}