
$ (10x^2+6xy+y^2=2)$ => $ ((x/\sqrt2)^{2} + ((3x+y)/\sqrt2))^{2} = 1 $
so if I change the variables to $u$ and $v$,
$u = x/\sqrt2$
$v= (3x+y)/\sqrt2) $
Then my bounds of integration become $-1 < v < 1$ and $-\sqrt{1-v^{2}} < u < \sqrt{1-v^{2}}$
The double integral is: $\int_{-1}^1 \int_{-\sqrt{1-v^2}}^{\sqrt{1-v^2}} (u\sqrt2)^{2} \frac12 \,du\,dv$
After a few steps of integration, I am left with:
$ \frac13 \int_{-1}^1 (1-v^2)^{3/2} dv$ which is pretty complicated.
Where did I go wrong?
You have $u=x/\sqrt{2}$ and $v= (3x+y)/\sqrt{2}$.
The $u$-axis is the set of points where $v=0$, and that is where $3x+y=0$. That's a line whose slope is $-3$. The $v$-axis is where $u=0$, and that's the same as $x=0$, so it's the same as the $y$-axis. Your problem is that
Thus $dx\,dy\ne du\,dv$. Since the transformation from $(x,y)$ to $(u,v)$ is linear, you'll have $dx\,dy\ne (\text{some constant} \cdot du\,dv)$. (If it were not linear, then the quantity you multiply by would not be constant, i.e. it would depend on $u$ and $v$.
That's one thing that Jacobians are used for.
Alternatively, you could use a different linear transformation that is actually a rotation, so that the constant is $1$.
PS: Draw the rectangle whose vertices are
You see a parallelogram whose area in the $xy$-plane is $2$. But in the $uv$-plane the area is $1$: it's just the area of a square. In other words an area of $1$ in the $uv$-plane corresponds to an area of $2$ in the $xy$-plane. That tells you that $dx\,dy=2\,du\,dv$.