So yes, this was asked before but I'm stuck on a specific step.
So to solve this, I tried to find the the intersection between the line $y=m(x-1)+4$ (we have $(1, 4)$ as a rational point on the circle) and $x^2 + y^2 = 17$. So simply substitution, $x^2 + (m(x-1)+ 4)^2 = 17 \implies m^2(x-1)^2 + 8m(x-1) + x^2 - 1 = 0$.
Now I want to find the roots to get all the rational points but I'm not sure how one would get the roots of this equation. Stuck on the algebra. Would appreciate help.
While you already have answers in one direction, I figure it's worth showing a different angle of approach to the problem: since 'by inspection' we have a particular solution $x=4, y=1$ to the problem, and using the sum-of-two-squares product identity, we can construct a map between solutions of $x^2+y^2=1$ and solutions of $x^2+y^2=17$.
In more detail: the equation $x^2+y^2=17$ is most easily thought about conceptually as $|z|^2=17$, with $z=x+iy$. Seen through this lens, it's clear that given our specific solution $z_0=4+i$, then we have $\left|\frac z{z_0}\right|^2=1$ whenever $|z|^2=17$; contrariwise, if $|w|^2=1$, then $|z_0w|^2=17$. Since the complex numbers form a field, these maps are bijections: if $z_0z=z_0w$ then $z=w$, and similarly if $\frac{z}{z_0}=\frac{w}{z_0}$. What's more, since $z_0$ is a 'rational' complex number, then the bijections take rationals to rationals.
But solving the equation $x^2+y^2=1$ in rationals the same as just solving the Pythagorean equation $a^2+b^2=c^2$, and its family of primitive solutions is well-established; you can use these to write explicit solutions to your equation.
(I'll add post-facto that this is really just an algebraic way of looking at the approach that you took; multiplication by $z_0$ is the same as rotating and scaling the complex plane, carrying the circle $|z|^2=1$ to the circle $|z|^2=17$, and carrying the point $(1,0)$ — i.e., $z=1$ — to the point $(4,1)$, which is just an $xy$ swap away from being the point that you were using for your projection, and one way of finding the canonical solutions to the Pythagorean equation is by projecting from the circle in much the same way as was done in the original post here. So the two approaches are really the same thing every step of the way, but I find the algebraic approach a little easier to understand.)