For which values of $p>0$ does the the double integral $$ \int \int_D \frac{1}{(x^4+2x^3y+3x^2y^2+3xy^3+y^2)^p} $$ where
$$D= {xy:x^2+y^2<1}$$
converge? $\textit{Tip: Try to use symmetries in the integrand to find substitutions to simpify the expression.}$
I have tried the following to simplify the denominator, but that doesn't seem to make the problem easier.
What could be a useful "symmetry in the integrand"?
$$x^4+2x^3y+3x^2y^2+2xy^3+y^4= \frac{(x+y)^4+x^4+y^4}{2}= \frac{((x+y)^2)^2+x^4+y^4}{2}= \frac{(x^2+y^2+2xy)^2+x^4+y^4}{2}$$
$\begin{cases} x = rcos(t)\\ y = rsin(t) \end{cases}$ gives
$$= \frac{(r^2+r^22cost(t)sin(t))^2+r^4cos^4(t)+r^4sin^4(t)}{2}=\frac{(r^2+r^2sin(2t))^2+r^4((cos^2(t)+sin^2(t))^2-2sin^2(t)cos^2(t))}{2}$$
$$=\frac{r^4(1+2sin(2t)+sin^2(2t)+1-\frac{1}{2}sin^2(2t))}{2}=\frac{r^4(2+2sin(2t)+\frac{1}{2}sin^2(2t)}{2}=r^4(1+sin(2t)+\frac{1}{4}sin^2(2t))$$
So this would result in
$$ \int_0^1 \frac{1}{r^3} \int_0^{2\pi} \frac{1}{(1+sin(2t)+\frac{1}{4}sin^2(2t))^p}drdt$$ but here I'm stuck..
As you pointed out, we can move to polar coordinates, yielding something of the form $$ \int_0^{2\pi} \int_0^1 \dfrac{g(\theta)}{r^3} dr d\theta = \int_0^{2\pi} g(\theta)\int_0^1\frac{1}{r^3} dr \, d\theta $$
Depending on how you are defining the convergence of improper double integrals, this may show that it does not converge (because one of the iterated integrals, for a particular order of integration, does not converge).