Problem. Evaluate $$\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}e^{-(5x^2-6xy+5y^2)}dxdy$$
My Solution. A hint is given that "Use $\int_{-\infty}^{\infty}e^{-ax^2}dx=\sqrt{\frac{\pi}{a}}$ "
Now,
$\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}e^{-(5x^2-6xy+5y^2)}dxdy=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}e^{-5(x-\frac{3y}{5})^2}e^{-\frac{16}{5}y^2}dxdy$
But here the range of integration is whole $\mathbb{R^2}$. I know the definition of double integral on a plane bounded region in $\mathbb{R^2}$. So How should I calculate this last integral?
Can I proceed by repeatedly integrate w.r.to $x$ and $y$ like the following:
$\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}e^{-5(x-\frac{3y}{5})^2}e^{-\frac{16}{5}y^2}dxdy=\int_{-\infty}^{\infty}e^{-\frac{16}{5}y^2}(\int_{-\infty}^{\infty}e^{-5(x-\frac{3y}{5})^2}dx)dy=\int_{-\infty}^{\infty}e^{-\frac{16}{5}y^2}(\sqrt{\frac{\pi}{5}})dy=\sqrt{\frac{\pi}{5}}\sqrt{\frac{\pi}{16/5}}=\frac{\pi}{4}$
Is this right approach? If so how?
$$ 5x^2-6xy-5y^2 \equiv 8X^2+2Y^2 $$
This is obtained making a change of variables (rotation) to eliminate the cross product $x y$
now
$$ \int_{-\infty}^{\infty}\int_{-\infty}^{\infty}e^{-(5x^2-6xy+5y^2)}dxdy = \int_{-\infty}^{\infty}\int_{-\infty}^{\infty}e^{-(8X^2+2Y^2)}dXdY $$
and then
$$ \int_{-\infty}^{\infty}e^{-8X^2}dX = \frac{\sqrt{\pi}}{2\sqrt 2}\\ \int_{-\infty}^{\infty}e^{-2Y^2}dY = \sqrt{\frac{\pi }{2}}\\ $$
hence
$$ \int_{-\infty}^{\infty}\int_{-\infty}^{\infty}e^{-(5x^2-6xy+5y^2)}dxdy =\frac{\pi}{4} $$