I have the following conic that contains an $xy$-term:
$$ x^2 + 4xy + y^2 + 8x + 4y - 2 = 0 $$
From what I have read, since an xy-term is present, the axes of the conic do not lie along the x- and y- axis, and the conic undergoes a rotation of $2\theta = \arctan(\frac{b}{a-c})$, where $a, b$ and $c$ are the coefficients of $x^2, xy,$ and $ y^2$ respectively.
However, from the above conic, I would end up with an angle of $2\theta = \arctan(\frac{4}{0})$. What would the angle of rotation be in this case?
I don't know where you got that formula from. Anyway, if$$f(x,y)=x^2+4xy+8x+y^2+4y-2,$$then$$f\left(\frac{X-Y}{\sqrt2},\frac{X+Y}{\sqrt2}\right)=3X^2+6\sqrt2 X-Y^2-2\sqrt2Y-2.$$Note that the map$$(X,Y)\mapsto\left(\frac{X-Y}{\sqrt2},\frac{X+Y}{\sqrt2}\right)$$is a counterclockwise rotation around the origin by an angle equal to $\frac\pi4$.