Solve in integers $x,y$ the equation $\left\lfloor\frac{xy-xy^2}{x+y^2} \right\rfloor=a$

Question

Solve in integers the equation $$\left\lfloor\frac{xy-xy^2}{x+y^2} \right\rfloor=a$$

My work so far:

1) If $a=1$, then $x\in\{-1;-2;-3\}$.

i) $x=-1 \Rightarrow y\ge-3$

ii) $x=-2 \Rightarrow y\ge3$

iii) $x=-3$. No solution.

2) $$a\le\frac{xy-xy^2}{x+y^2}<a+1$$

Answer 1

This answer shows a way to find solutions and all solutions for $|a|\le 3$.

We have $$a=\left\lfloor\frac{xy-xy^2}{x+y^2}\right\rfloor=\left\lfloor-x+\frac{x^2+xy}{x+y^2}\right\rfloor=-x+\left\lfloor\frac{x^2+xy}{x+y^2}\right\rfloor$$

So, if we write $$\left\lfloor\frac{x^2+xy}{x+y^2}\right\rfloor=k$$ then $y$ is an integer such that there exists an integer $k$ satisfying $$k\le \frac{(k-a)^2+(k-a)y}{(k-a)+y^2}\lt k+1\tag1$$ and for such $(y,k)$, $x=k-a$.

In the following, let us separate it into cases.

Case 1 : $k-a+y^2\gt 0$

$$\begin{align}(1)&\iff k(k-a+y^2)\le (k-a)^2+(k-a)y\lt (k+1)(k-a+y^2)\\&\iff ky^2\le -ka+a^2+ky-ay\lt ky^2+k-a+y^2\\&\iff k(y^2-y+a)\le a^2-ay\quad\text{and}\quad k(y^2-y+a+1)\gt a^2-ay+a-y^2\end{align}$$

Case 1-1 : $y^2-y+a\le -2$ $$\frac{a^2-ay}{y^2-y+a}\le k\lt \frac{a^2-ay+a-y^2}{y^2-y+a+1}$$

However, we have $$(a^2-ay+a-y^2)(y^2-y+a)\gt (a^2-ay)(y^2-y+a+1)\iff y^3(y-1)\lt 0$$ There is no such $y\in\mathbb Z$.

Case 1-2 : $y^2-y+a=-1$ $$-k\le a^2-ay\quad\text{and}\quad 0\gt a^2-ay+a-y^2$$ $$\iff -k\le a^2-ay\quad\text{and}\quad 0\gt (-1-y^2+y)^2-(-1-y^2+y)y+(-1-y^2+y)-y^2$$ There is no such $y\in\mathbb Z$.

Case 1-3 : $y^2-y+a=0$

$$0\le (-y^2+y)^2-(-y^2+y)y\quad\text{and}\quad k\gt (-y^2+y)^2-(-y^2+y)y+(-y^2+y)-y^2$$ The first inequality holds for any $y\in\mathbb Z$.

Case 1-4 : $y^2-y+a\ge 1$

$$\frac{a^2-ay+a-y^2}{y^2-y+a+1}\lt k\le \frac{a^2-ay}{y^2-y+a}\tag2$$ We have $$(a^2-ay+a-y^2)(y^2-y+a)\lt (a^2-ay)(y^2-y+a+1)$$$$\iff y^3(y-1)\gt 0\implies y\not=0,1$$

$k=0$ is the only integer satisfying $(2)$ when $y$ satisfies $$-1\le \frac{a^2-ay+a-y^2}{y^2-y+a+1}\lt 0\quad\text{and}\quad 0\le \frac{a^2-ay}{y^2-y+a}\lt 1$$ $$\iff -y^2+y-a-1\le a^2-ay+a-y^2\lt 0\quad\text{and}\quad 0\le a^2-ay\lt y^2-y+a$$ $$\small \iff (a+1)y\le (a+1)^2\quad \text{and}\quad y^2+ay-a^2-a\gt 0\quad\text{and}\quad ay\le a^2\quad\text{and}\quad y^2+(a-1)y-a^2+a\gt 0$$

$k=-1$ is the only integer satisfying $(2)$ when $y$ satisfies $$-2\le \frac{a^2-ay+a-y^2}{y^2-y+a+1}\lt -1\quad\text{and}\quad -1\le \frac{a^2-ay}{y^2-y+a}\lt 0$$ $$\iff -y^2+2y-2a-2\le a^2-ay+a\lt y-a-1\quad\text{and}\quad -y^2+y-a\le a^2-ay\lt 0$$ $$\small \iff (a+1)y\gt (a+1)^2\quad \text{and}\quad y^2+(-2-a)y+a^2+3a+2\ge 0\quad\text{and}\quad ay\gt a^2\quad\text{and}\quad y^2+(-a-1)y+a^2+a\ge 0$$

From these, we know that the number of $y$ we need to see is finite in this case 1-4.

Case 2 : $k-a+y^2\lt 0$

$$\begin{align}(1)&\iff k(k-a+y^2)\ge (k-a)^2+(k-a)y\gt (k+1)(k-a+y^2)\\&\iff ky^2\ge -ka+a^2+ky-ay\gt ky^2+k-a+y^2\\&\iff k(y^2-y+a)\ge a^2-ay\quad\text{and}\quad k(y^2-y+a+1)\lt a^2-ay+a-y^2\end{align}$$

Case 2-1 : $y^2-y+a\le -2$

$$\frac{a^2-ay+a-y^2}{y^2-y+a+1}\lt k\le \frac{a^2-ay}{y^2-y+a}$$

Case 2-2 : $y^2-y+a=-1$

$$-k\ge a^2-ay\quad\text{and}\quad 0\lt (-1-y^2+y)^2-(-1-y^2+y)y+(-1-y^2+y)-y^2$$ So, $y\not=0, y\not=1$.

Case 2-3 : $y^2-y+a=0$

$$0\ge (-y^2+y)^2-(-y^2+y)y\quad\text{and}\quad k\lt a^2-ay+a-y^2$$ So, $y=0,1$. For $y=0$, $a=0$ and $k\lt 0$. For $y=1$, $a=0$ and $k\lt -1$.

Case 2-4 : $y^2-y+a\ge 1$

$$\frac{a^2-ay}{y^2-y+a}\le k\lt \frac{a^2-ay+a-y^2}{y^2-y+a+1}$$

However, we have $$(a^2-ay+a-y^2)(y^2-y+a)\gt (a^2-ay)(y^2-y+a+1)\iff y^3(y-1)\lt 0$$ There is no such $y\in\mathbb Z$.

So, we know that case 1-1, case 1-2 and case 2-4 don't happen.

---

In each case, $y$ is determined by a given $a$, and for each such $y$ (the number of $y$ you need to see is only finite in each case), see if there exists an integer $k$ satisfying the conditions. If you have such $(y,k)$, then you get $(x,y)$ where $x=k-a$.

All solutions for $|a|\le 3$ :

$a=-3$ : $(x\le -12,y=2), (-11\le x\le -3, y=-1), (x=2, y\le -3), (x=3, y=-3,-2,y\ge 5), (x=4, y=-2, 4,5,6), (x=5, y=3,4),(6\le x\le 9, y=3)$

$a=-2$ : $(x=1, y\le -2), (x=2,y=-2,-1, y\ge 3), (x=3, y=-1,3,4), (x=4, y=-1,3), (x\ge 5, y=2)$

$a=-1$ : $(x=1,y=-1,y\ge 2), (2\le x\le 4,y=2)$

$a=0$ : $(x\le -2,y=0,1), (x=-1, y=0,y\ge 2), (x=0,y\le -1,y=1,y\ge 2), (x\ge 1, y=0,1)$

$a=1$ : $(x=-2,y\ge 3),(x=-1,y\le -3)$

$a=2$ : $(x=-3,y\ge 4),(x=-2,y\le -4,y=2),(x=-1,y=-2)$

$a=3$ : $(x=-4, y\ge 5), (x=-3,y\le -6,y=3), (x=-2,y=-3)$

Answer 2

HINT.-$x+y^2\ne 0$. Consider two cases $x+y^2\gt 0$ and $x+y^2\lt 0$.

$$\boxed{x+y^2\gt 0}$$

You have $$a(x+y^2)\le xy-xy^2\lt a(x+y^2)+(x+y^2)$$ This gives two quadratic inequalities in $y$ $$\begin{cases}(a+x)y^2-xy+ax\le 0\\(a+x+1)y^2-xy+ax+x\gt 0\end{cases}$$ Now for $x$ fixed, $(\alpha, \beta)$ and $(\gamma,\delta)$ being the corresponding roots supposed reals and $a+x\gt 0$ you have for the values of $y$ the restrictions $$\alpha\le y\le \beta\\y\in]-\infty,\gamma[\space\cup \space]\delta,+\infty[$$ You can see that the roots play a strong role in this and the discusion of possibilities certainly don't stop here. As you say in your comment "Perhaps the problem is not of a "normal" solutions".