I came across this question in the Resonance Journal of science education (April edition 2021). Unfortunately it is only available in the hard copy of the magazine. There is a small poem cited from $The \ Ladies' \ Diary \ 1776$ that proposes the following problem,
$x^2+xy+y^2=1087$
$x^4+x^3y^3+y^4=45777295$
I was able to find a cubic equation in xy by substituting the value of $x^4 + y^4$ from first equation into the second equation,
$(xy)^3-(xy)^2-2174(xy)-44595726=0$
Now the questions I have is,
- Is this the correct way to approach such a problem? If not please provide an alternative approach.
- How do I proceed from here? I prime factorised the last term into $44595726=2×3×7×17×62459$, now do I need to consider all possible combinations of products of these prime factors and plug it in the equation to solve it? (Assuming of course that there were no calculators nor polynomial equation solvers in 1776)

The second equation you are given has a subtle hint.
If $x$ and $y$ are large, then the $x^3y^3$ which is degree 6 dominates the other terms.
We can establish this dominance more rigorously by observing that the first given equation forces $x,y\le32$. Then $x^4+y^4$ cannot exceed $2×32^4=2097152$, which is less than $5$% of the right side of the second equation, forcing the $x^3y^3$ term to make up all the rest.
So $xy$ should be close to the cube root of the right side of the second equation, this root being between $300$ and $400$.
The factor that fits this estimate and your product is $3×7×17=357$, so try that value for $xy$.
Assuming that $xy=357$ works (solving the cubic equation), the first equation then gives $x^2+y^2=730$. Then the larger factor must be at least $\sqrt{xy}=\sqrt{357}$ and less than $\sqrt{x^2+y^2}=\sqrt{730}$. The only factor of $357$ that falls within these limits is $21$, so the values of $x$ and $y$ would be (in either order) $21$ and $357/21=17$.