I have a quartic polynomial of shape $$(a\cdot x+b+c\cdot xy-d\cdot y+y^2\big)^2$$ where $a,b,c,d\in\mathbb Z_{>0}$ are coefficients.
The polynomial is not convex.
However is it quasiconvex for various values of $(a,b,c,d)$?
My problem is to find the regions of $(a,b,c,d)\in\mathbb Z_{>0}^4$ on which the polynomial is quasiconvex?
Note this polynomial is always non-negative and expressible as sum of squares. If so may be explicit representation will help. Note $0$ level is always the lower bound. It might be easy to handle.
Let $g(x,y) = ax+b+cxy-dy+y^2$ so that $f(x,y) = (ax+b+cxy-dy+y^2)^2 = g(x,y)^2$. If $g$ has a non-convex zero set, then $f$ won't be quasi-convex - for small enough $\epsilon >0$, $f^{-1}((-\infty,\epsilon))$ will be a neighborhood of $g^{-1}(0)$ which will again be nonconvex. So our only hope of $f$ being quasiconvex is when $g=0$ is convex.
Our goals are for this conic to be a single line, a single point, or empty, as these are the only conics which are convex. We can read off when $ax+b+cxy-dy+y^2=0$ is of the appropriate form by examining the matrix form of our conic, $$G =\begin{pmatrix} 0 & c/2 & a/2 \\ c/2 & 1 & -d/2 \\ a/2 & -d/2 & b \end{pmatrix}$$
Let $U$ be the upper left $2\times 2$ minor.
In order to get a single line, a single point, or an empty conic, we must have $\det G =0$ and $\det U \geq 0$. On the other hand, $\det U = \frac{-c^2}{4}$, which as $c\neq 0$ by the problem statement, is always negative, so $f$ is never quasiconvex.