When $x^2+6xy+y^2$ a square number?

Question

Find all natural numbers $x$ and $y$ such that $x^2+6xy+y^2$ is a square number.

For example, $(x,y)=(2,3)$ or $(x,y)=(3,10)$.

Obviously, we can consider $gcd(x,y)=1$.

Answer 1

Stereographic projection from $(-1,0),$ I get $$ x = m^2 - n^2, \; \; y = 2 mn + 6 n^2, $$ with $$ \gcd(m,n) = 1, $$ and either $m > n > 0$ or $n < 0$ and $|n| < m < 3 |n|. $ Also $m,n$ not both odd.

Put the inequalities together, we get $$ m > n > 0 \; \; \mbox{OR} \; \; \frac{-m}{3} >n > -m. $$ In the latter case we are originally in the third quadrant as the rational point on the hyperbola $x^2 + 6xy+y^2 = 1$ is $$ x= \frac{m^2 - n^2}{m^2 + 6 mn+n^2}, \; \; y= \frac{2mn+6n^2}{m^2 + 6 mn+n^2}, $$ while $-m/3 > n > -m$ implies $m^2 + 6 mn + n^2 < 0.$

Improvement: If $m,n$ both odd, take $$ x = \frac{m^2 - n^2}{4}, \; \; y = \frac{2 mn + 6 n^2}{4}, $$ final requirement here is that $m \neq n \pmod 4.$ Put another way: when both are odd, we require that $m+n$ be divisible by $4.$

The nonsense with the possible negative signs comes from the fact that we are not projecting onto an ellipse, we are projecting onto a hyperbola. So, the denominator may come up negative and solutions disappear. Need to allow $n$ negative. Let me know if I have missed any solutions; fairly easy to just splat a formula on the page, harder to figure out whether one has all solutions.

Note that, given some pair $(x,y)$ such that $x^2 + 6 xy+ y^2 = w^2$ for some $w,$ we get new pairs ( same $w$) with $$ (-y,x+6y), $$ $$ (-x-6y,6x+35y), $$ $$ (-6x-35y,35x+204y), $$ $$ (-35x-204y,204x+1189y) $$ and so on, which pushes along the relevant hyperbola in the second or fourth quadrant. As you will see, there is some repetition below. First ordered by $m,n$ then a short thing ordered by just $x,y.$

EDIT: this worked out very well. I just restricted to those solutions with $x>y$ and put them in order, well, backwards...Being in order, I was able to compare with a list of all solutions with $x \leq 306,$ we have a winner.

 x      y                m       n
306     19               35      1
300     13               37    -13
297     80               19     -8
290    171               39    -19
286    279               35      9
285     92               17      2
275     42               18     -7
273    232               17      4
270    119               37    -17
261    220               19    -10
260    189               33      7
255     38               16      1
253     12               17     -6
247    150               16      3
240     17               31      1
234    115               31      5
230     39               33    -13
225    112               17     -8
221     84               15      2
216    209               35    -19
210     11               31    -11
208     57               29      3
207     70               16     -7
200    153               33    -17
198    175               29      7
195     34               14      1
187    138               14      3
184    105               31    -15
182     15               27      1
176    105               27      5
171     10               14     -5
168     65               29    -13
165     76               13      2
161    144               15     -8
154     51               25      3
152     33               27    -11
143     30               12      1
136      9               25     -9
133     60               13     -6
132     13               23      1
126     95               23      5
119     30               12     -5
117     68               11      2
114     91               25    -13
105      8               11     -4
102     55               23    -11
 99     26               10      1
 90     11               19      1
 85     84               11     -6
 78      7               19     -7
 77     60                9      2
 70     39               17      3
 65     24                9     -4
 63     22                8      1
 56      9               15      1
 55      6                8     -3
 52     45               17     -9
 44     21               15     -7
 40     33               13      3
 36      5               13     -5
 35     18                6      1
 30      7               11      1
 21      4                5     -2
 15     14                4      1
 12      5                7      1
 10      3                7     -3
  3      2                2     -1

      x      y                m      n
      3      2                2     -1
      3     10                2      1
      5     12                3     -2
      2      3                3      1
      5     36                3      2
      7     30                4     -3
     15     14                4      1
      7     78                4      3
      9     56                5     -4
     21      4                5     -2
     21     44                5      2
      4     21                5      3
      9    136                5      4
     11     90                6     -5
     35     18                6      1
     11    210                6      5
     13    132                7     -6
     33     40                7     -4
     10      3                7     -3
     12      5                7      1
     45     52                7      2
     33    152                7      4
      6     55                7      5
     13    300                7      6
     15    182                8     -7
     39     70                8     -5
     55      6                8     -3
     63     22                8      1
     55    102                8      3
     39    230                8      5
     15    406                8      7
     17    240                9     -8
     14     15                9     -5
     65     24                9     -4
     77     60                9      2
     65    168                9      4
      8    105                9      7
     17    528                9      8
     19    306               10     -9
     51    154               10     -7
     99     26               10      1
     91    114               10      3
     51    434               10      7
     19    666               10      9
      x      y                m      n

         x         y
         1         0
         3         2
        10         3
        12         5
        15        14
        21         4
        30         7
        35        18
        36         5
        40        33
        44        21
        52        45
        55         6
        56         9
        63        22
        65        24
        70        39
        77        60
        78         7
        85        84
        90        11
        99        26
         x         y
jagy@phobeusjunior: 

I was curious about repetition, still with positive $x,y,$ to $x^2 + 6 xy+ y^2 = z^2.$ Plenty, and predictable, squarefree products of primes $p \equiv \pm 1 \pmod 8.$

     x      y                m      n               z
     65     24                9     -4             -119    7 . 17 
     90     11               19      1              119  

     99     26               10      1              161    7 . 23
    136      9               25     -9             -161

    143     30               12      1              217   7 . 31
    102     55               23    -11             -217

    351     14               20     -7             -391   17 . 23
    176    105               27      5              391

   2628     37              109    -37            -2737   7 . 17 . 23
   1927    318               44      3             2737
   1333    660               37      6             2737
   1025    912               33      8             2737

  81807   1030              304   -103           -84847   7 . 17 . 23 . 31
  62625   8528              271   -104           -84847
  59998   9735              491     33            84847
  55040  12177              471     41            84847
  49773  15052              247   -106           -84847
  45140  17877              429     59            84847
  39195  22018              226   -109           -84847
  35032  25353              383     81            84847
     x      y                m      n               z

Answer 2

For non-trivial cases, $xy\ne0$

Let $x^2+6xy+y^2=(x+ky)^2$ where $k$ is any integer

$\iff y(6x+y)=y(2kx+k^2y)\implies6x+y=2kx+k^2y\iff x(6-2k)=y(k^2-1)$

So, $\dfrac x{k^2-1}=\dfrac y{6-2k}=m$(say an integer)

As $x,y>0$ if $m<0,$ we need $6-2k<0\iff k>3\ \ \ \ (1)$ and $k^2-1<0\iff -1<k<1\ \ \ \ (2)$

There can be no $k$ satisfying both $(1),(2)$

Similarly if $m>0,$ we need $6-2k>0\iff k<3$ and $k^2-1>0\iff k>1$ or $k<-1$

$k<3,k<-1\implies k<-1$

or $k<3,k>1\implies1<k<3\implies k=2$

So, there are infinitely natural values of $x,y$