Why does this double integral evaluate to different results when evaluated with polar coordinates?

Question

$ \int_0^{1}\int_0^1 xy\sqrt{x^2 + y^2} dxdy $

When switched over to polar coordinates, the above integral will look like the following,

$ \int_0^{\pi/4}\int_0^\sqrt{2} \frac{r^4sin(2\theta)}{2} drd\theta $

But the first integral evaluates to $ \frac{2}{15}(2\sqrt(2)-1) $ whereas the second integral evaluates to $ \frac{2\sqrt{2}}{5} $

Answer 1

Actually, that integral, in polar coordinates, is$$\color{red}{\int_0^{\pi/4}\int_0^{1/\cos(\theta)}\rho^4\cos(\theta)\sin(\theta)\,\mathrm d\rho\,\mathrm d\theta}+\color{blue}{\int_{\pi/4}^{\pi/2}\int_0^{1/\sin(\theta)}\rho^4\cos(\theta)\sin(\theta)\,\mathrm d\rho\,\mathrm d\theta},\label{a}\tag1$$where the red and the blue parts of \eqref{a} correspond to the integral of $xy\sqrt{x^2+y^2}$ over the red and blue triangles of this picture:

And it turns out that \eqref{a} is equal to$$\frac2{15}\left(2\sqrt{2}-1\right).$$

Answer 2

The set $\{ (x,y) \mid 0 \le x,y \le 1\}$ graphs the unit square in Cartesian coordinates:

In polar coordinates $\{ (r,\theta) \mid 0 \le r \le \sqrt 2, 0 \le \theta \le \pi/4 \}$ graphs $1/8$ of a circle of radius $\sqrt 2$, centered at the origin:

When changing bounds for integrals like this, you need to make sure that, before and after, the bounds make the same shape. Here, they do not, so it is no surprise you get different integrals.