The question asks for positive integer solutions to $x^2+y^2+x+y+1=xyz$ .
We at first note that $x|y^2+y+1$. Now,let there exist positive integers $x,y$ that satisfy the given equation.Then $mx=y^2+y+1$ for some positive integer $m$ .Substituting this in the original equation yields
$$x^2+x+mx=xyz$$ $$\implies x+1+m=yz$$ $$\implies \dfrac{y^2+y+1}{m}+1+m=yz$$ $$\implies y^2+y+m^2+m+1=myz$$ This equation's exactly like the equation we were trying to solve.Unfortunately,however,this does not let us use FMID since we don't know if $m<x$.
The above problem is from Titu Andreescu's book on intro to diophantine equations.He does give a solution,but I was wondering if my approach could be used somehow.I am also interested in seeing other approaches.Thanks is advance.
This is a classic example of a problem which is easily solved via Vieta jumping.
We may WLOG assume $x \geq y$. Let us define the "size" of a positive integer solution $(x, y, z)$ to be $x+y \in \mathbb{Z}^+$, and say that a solution is a smaller solution if its size is smaller.
Given a positive integer solution $(x, y, z)$ with $x>y\geq 1$, we shall use Vieta's formulas to obtain a smaller solution $(y, x', z)$ with $x>y \geq x' \geq 1$ (so $x+y>y+x'$).
Indeed, suppose we have a positive integer solution $(x, y, z)$ with $x>y \geq 1$. Rewrite the equation as $x^2+(1-yz)x+(y^2+y+1)=0$. Thus $x$ is a root of the quadratic $t^2+(1-yz)t+(y^2+y+1)=0$.
Let $x'$ be the other root. Vieta's formulas give $x+x'=yz-1, xx'=y^2+y+1$. The sum of roots equation tells us that $x'$ is an integer (since $x, yz-1$ are integers), and the product of roots equation tells us that $x'>0$ (since $x, y^2+y+1>0$), so $x'$ is indeed a positive integer. Thus we get a positive integer solution $(y, x', z)$.
Since $x>y$, we have $x'=\frac{y^2+y+1}{x} \leq \frac{y^2+y+1}{y+1}<y+1$ so $x' \leq y<x$, as desired. Thus $(y, x', z)$ is indeed a smaller positive integer solution. (where smaller means $y+x'<x+y$)
Note: Here $x'$ is essentially the same as your $m$; note $x'=\frac{y^2+y+1}{x}$.
Now, starting from any positive integer solution $(x, y, z)$, we will "jump down" to a smaller positive integer solution as long as $x>y$. Since this cannot be done indefinitely (Every non-empty subset of the positive integers has a least element), we must eventually jump down to a base solution where $x=y$.
When $x=y$, we get $2y^2+2y+1=y^2z$ so $y \mid 1$ so $y=1$, so $z=5$. Thus the only base solution is $(x, y, z)=(1, 1, 5)$. Now, if we start from an arbitrary positive integer solution, then after jumping down some finite number of times, we must reach a base solution, which we have shown must be $(1, 1, 5)$. Thus all positive integer solutions of the orginal equation may be recovered by "jumping back up", i.e. reversing the process of getting a smaller solution explained above.
Note that $z$ remained constant throughout the process of jumping, so we have $z=5$ throughout. We have $x+x'=yz-1=5y-1$, so we may reverse the jump and "jump up" from $(y, x', 5)$ to $(5y-1-x', y, 5)$.
Thus all positive integer solutions are given by $(x_n, y_n, 5)$ where $x_1=y_1=1$, and $(x_{n+1}, y_{n+1}, 5)=(5x_n-1-y_n, y_n, 5)$. If we define an auxillary recurrence $a_1=a_2=1, a_{n+1}=5a_n-1-a_{n-1}$ for $n \geq 2$, then all positive integer solutions are given by $(x, y, z)=(a_{n+1}, a_n, 5)$, where $n \in \mathbb{Z}^+$. The first few terms of the recurrence $a_n$ are $1, 1, 3, 13, 61, 291$. Checking in OEIS gave this exact problem.
Oh, and if you relax the assumption $x \geq y$, you just get $(x, y, z)=(a_{n+1}, a_n, 5)$ and $(x, y, z)=(a_n, a_{n+1}, 5)$