Let $f(x, y)$ be the given function. $$f(x, y) = ce^{-(x^2 - xy + 4y^2)/2}$$ Determine the value of $c$ such that $f$ is a pdf.
The problem is finding the value of $c$ such that
$$ \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} ce^{-(x^2 - xy + 4y^2)/2}\,dx\,dy = 1 $$
The usual way of integrating a regular gaussian/normal pdf is by using polar coordinates. However, the values on the exponential above make it harder than usual. I assume there must be some transformation of variables that will make the integral simpler, then you'd need to calculate the jacobian obviously, but I can't see the transformation.
It suffices to "mould" your expression onto the multivariate normal distribution formula one can find for example here, with $m=0$ (mean = $0$) and $n=2$ ($n$ = number of variables).
We have, in our case :
$$c \exp(-\dfrac12 (X^T S X))=c \exp(-\dfrac12 (X^T \Sigma^{-1} X))$$
where $X=\binom{x}{y}$ and :
$$S=\begin{pmatrix}1&-\tfrac12\\ -\tfrac12&4\end{pmatrix} \ \implies \ \Sigma=S^{-1}=\tfrac{16}{15}\begin{pmatrix}4&\tfrac12\\ \tfrac12&1\end{pmatrix}$$
($\Sigma$ is the variance matrix extending the concept of unidimensional variance), giving :
$$c=\dfrac{1}{2 \pi \sqrt{\det(\Sigma)}}$$
But in fact, one can completely shortcut the computation of $\Sigma$. Indeed $\Sigma=S^{-1} \ \implies \ \det(\Sigma)=\det(S)^{-1}$, allowing to write plainly :