I calculated density function for (X, Y): $\frac{1}{2\sqrt{3}\pi}e^{-\frac{x^2-xy+y^2}{3}}$ But if i try to write probability function via integral, it becomes to hard to integrate:
$$\int_{v > \frac{u}{\operatorname{tg}(t)}} e^{-\frac{u^2}{3}}e^{\frac{vu}{3}}e^{-\frac{v^2}{3}}$$
And after integration I'll need to differentiate it to find density...
It is tempting to utilize the polar transformation: \begin{equation} \begin{cases} X = R\cos\Theta, \\ Y = R\sin\Theta. \end{cases} \end{equation} Then $\arctan(Y/X) = \Theta$. To find the distribution of $\Theta$, it is standard to start with the joint distribution of $(R, \Theta)$. The Jacobian is \begin{equation} \frac{\partial(x, y)}{\partial(r, \theta)} = r \end{equation} is standard. Hence the joint density of $(R, \Theta)$ is \begin{align} f(r, \theta) = \frac{r}{2\sqrt{3}\pi}\exp\left(-\frac{1}{3}(r^2 - r^2\sin\theta\cos\theta)\right), \quad r > 0, \theta \in (0, 2\pi]. \end{align} Can you take it from here?