Show that if $a,b,c\in \mathbb{N}$ and ${a^2+b^2+c^2}\over{abc+1}$ is an integer it is the sum of two nonzero squares

Question

I was reading about the fascinating problem in the IMO ($1988 $ #$6$) that asks:

Let $a$ and $b$ be positive integers such that $(ab+1) | (a^2+b^2)$. Show that ${a^2+b^2}\over{ab+1}$ is a perfect square.

I wondered what would happen with the natural extension of this problem to three constants. Upon exploration with Mathematica, I found that the integral values of ${a^2+b^2+c^2}\over{abc+1}$ all seemed to be expressible as the sum of two nonzero perfect squares. The converse also seems to be true (given that a number is the sum of two nonzero squares, it seems to be expressible in the form of this fraction).

So I pose two questions:

If $a,b,c\in \mathbb{N}$ and $k={{a^2+b^2+c^2}\over{abc+1}}$ is an integer, is $k$ the sum of two nonzero squares?

and

Given that $k$ is expressible as the sum of two nonzero perfect squares, do there exist $a,b,c\in\mathbb{N}$ for which ${{a^2+b^2+c^2}\over{abc+1}}=k$?

Answer 1

I PROVED THE CONJECTURE

$$ x^2 + y^2 + z^2 = k + kxyz $$ Notice that we cannot have a solution with $x<0$

Got the other part. Let $x \geq y \geq z \geq 1$ be a solution giving $k$ that minimizes $x+y+z$ among positive solutions giving $k.$ Vieta jumping is just the new solution $$ (x,y,z) \mapsto (kyz-x,y,z) $$ The equation is $$ x^2 - (kyz)x + y^2 + z^2 - k = 0. $$ The replacement root $x'$ gives $x + x' = kyz. $ The product is $xx' = y^2 + z^2 - k.$

We have minimized $x+y+z$ among positive triples giving $k.$ We get a new triple with $x' = kyz - x.$ As it is impossible to have $x' <0,$ we have an Alternative:

(A) $kyz - x = 0$

(B) $kyz - x \geq x$

We will show that alternative (B) does not occur for, say, $k > 2.$

ASSUME (B), namely $kyz \geq 2x, $ with $x \geq y \geq z \geq 1.$ $$ x^2 + y^2 + z^2 = k + (kyz)x \geq k + (2x)x= k + 2x^2 $$ $$ \color{red}{ y^2 + z^2 \geq k + x^2}. $$ But $$ y^2 - (kx)yz + z^2 = k - x^2. $$ Add, $$ 2y^2 - (kx)yz + 2z^2 \geq 2k . $$ $$ y^2 - \left(\frac{kx}{2} \right)yz + z^2 \geq k . $$ So, $y \geq z \geq 1$ and $$ \color{red}{y^2 - \left(\frac{kx}{2} \right)yz + z^2 > 0 }. $$ From the quadratic formula, $$ y > \frac{1}{4} \left( kx + \sqrt {k^2 x^2 - 16} \right) z . $$ When $k \geq 4,$ we see $y > x,$ a contradiction. When $k=3$ and $z \geq 2,$ we still get $y > x.$ Finally, with $k=3$ and $z=1,$ the condition $y^2 + z^2 \geq k + x^2$ reads $y^2 + 1 \geq 3 + x^2$ or $y^2 \geq 2 + x^2,$ so again $y > x.$

These contradictions tell us that alternative (A) actually holds for $k \geq 3.$ That is, $kyz = x,$ so $kyz - x = 0,$ and we have a new solution $(0,y,z)$ with $y \geq z \geq 1$ and this $x=0$ to $$ x^2 + y^2 + z^2 = k + kxyz. $$ As this $x=0,$ we have reached $$ \color{red}{ y^2 + z^2 = k } $$

Answer 2

The method is called Vieta Jumping. It is a special case of the automorphism group of an indefinite binary quadratic form.

Part Two

The first part of your question: given $k = a^2 + b^2,$ you get an answer with $$ x=a, \; \; y =b, \; \; z = kab $$ Then $$ \frac{x^2 + y^2 + z^2}{1+xyz} = k $$

Part One

The description of Vieta Jumping in links is not quite specific enough to settle this. The conjecture seems very likely; here are the values of $k$ with $ 5000 \geq a \geq b \geq c \geq 1,$ but ignoring the many, many ways to get $k=2.$

     k         a         b         c         k
     5        10         2         1         5 =  5
     5      1102       230         1         5 =  5
     5       230        48         1         5 =  5
     5      2399        48        10         5 =  5
     5        48        10         1         5 =  5
     5      4948        99        10         5 =  5
     5       980        99         2         5 =  5
     5        99        10         2         5 =  5
     8        32         2         2         8 =  2^3
     8       510        32         2         8 =  2^3
    10      2940       297         1        10 =  2 5
    10       297        30         1        10 =  2 5
    10        30         3         1        10 =  2 5
    10       899        30         3        10 =  2 5
    13      2025        78         2        13 =  13
    13      3040        78         3        13 =  13
    13        78         3         2        13 =  13
    17      1152        68         1        17 =  17
    17      4623        68         4        17 =  17
    17        68         4         1        17 =  17
    18       162         3         3        18 =  2 3^2
    20       160         4         2        20 =  2^2 5
    25       300         4         3        25 =  5^2
    26       130         5         1        26 =  2 13
    26      3375       130         1        26 =  2 13
    29       290         5         2        29 =  29
    32       512         4         4        32 =  2^5
    34       510         5         3        34 =  2 17
    37       222         6         1        37 =  37
    40       480         6         2        40 =  2^3 5
    41       820         5         4        41 =  41
    45       810         6         3        45 =  3^2 5
    50      1250         5         5        50 =  2 5^2
    50       350         7         1        50 =  2 5^2
    52      1248         6         4        52 =  2^2 13
    53       742         7         2        53 =  53
    58      1218         7         3        58 =  2 29
    61      1830         6         5        61 =  61
    65      1820         7         4        65 =  5 13
    65       520         8         1        65 =  5 13
    68      1088         8         2        68 =  2^2 17
    72      2592         6         6        72 =  2^3 3^2
    73      1752         8         3        73 =  73
    74      2590         7         5        74 =  2 37
    80      2560         8         4        80 =  2^4 5
    82       738         9         1        82 =  2 41
    85      1530         9         2        85 =  5 17
    85      3570         7         6        85 =  5 17
    89      3560         8         5        89 =  89
    90      2430         9         3        90 =  2 3^2 5
    97      3492         9         4        97 =  97
    98      4802         7         7        98 =  2 7^2
   100      4800         8         6       100 =  2^2 5^2
   101      1010        10         1       101 =  101
   104      2080        10         2       104 =  2^3 13
   106      4770         9         5       106 =  2 53
   109      3270        10         3       109 =  109
   116      4640        10         4       116 =  2^2 29
   122      1342        11         1       122 =  2 61
   125      2750        11         2       125 =  5^3
   130      4290        11         3       130 =  2 5 13
   145      1740        12         1       145 =  5 29
   148      3552        12         2       148 =  2^2 37
   170      2210        13         1       170 =  2 5 17
   173      4498        13         2       173 =  173
   197      2758        14         1       197 =  197
   226      3390        15         1       226 =  2 113
   257      4112        16         1       257 =  257
   290      4930        17         1       290 =  2 5 29
    k         a          b         c        k