number of solutions of a quadric diophantine equation

Question

In what cases the quadratic diophantine equation $ax^2+bxy+cy^2=n$(n is a given number) has an infinite number of solution

Answer 1

When $b^2 - 4ac > 2$ but $b^2 - 4ac $ is not a square, and there is at least one integer solution $(x,y)$ to $ax^2 + bxy + cy^2 = n,$ then there are infinitely many solutions for that $n.$

Indeed, let $\Delta = b^2 - 4ac$ and $(\tau, sigma)$ the smallest solution with $\tau > 0, \sigma > 0$ to $$ \tau^2 - \Delta \sigma^2 = 4, $$ for each $x,y$ you get the next solution, and the next after that, with $$ (x,y) \mapsto \left( \left( \frac{\tau - b \sigma}{2} \right) x - c \sigma y, \; \; a \sigma x + \left( \frac{\tau + b \sigma}{2} \right) y \right)$$

Caution: unless $n = \pm 1,$ we expect at least two such "orbits" of solutions, with no explicit upper bound on the number of orbits if $n$ has many distinct prime factors. Still for each fixed $n,$ a finite number of orbits. This is how Siegel's approach to The Mass Formula for quadratic forms works out in the indefinite binary case. Well, there is a good deal more to it...

Note that $$ \left( \begin{array}{rr} \frac{\tau - b \sigma}{2} & -c \sigma \\ a \sigma & \frac{\tau + b \sigma}{2} \end{array} \right) $$ has determinant $1$ and trace $\tau.$

We can then construct separate linear recurrences for $x$ and $y$ using Cayley-Hamilton, $$ x_{k+2} = \tau \, x_{k+1} - x_k, $$ $$ y_{k+2} = \tau \, y_{k+1} - y_k. $$