I need to do the following:
Prove that a quadratic expression of the form $A(x^2-y^2) - (B-C)xy$ can be always factored into two linear factors.
It is easy enough to compare this with the standard representation $ax^2 + 2hxy + by^2 + 2gx + 2fy + c$ and say that the condition $abc + 2fgh -af^2 - bg^2 - ch^2 = 0$ is satisfied.
However, what are those two linear factors?
At best the given expression can be written as $Ax^2 - Ay^2 - Bxy + Cxy$ or $x(Ax-By) -y(Ax-Cy)$. But this doesn't look like it can be always factored.
Second try: Let $B-C = D$ for convenience.
Now the expression is:
$Ax^2 - Ay^2 - Dxy$
$= Ax^2 - (Dy)x + Ay^2$
If this be equated to zero,
$x = \dfrac{Dy \pm \sqrt{(Dy)^2 - 4A^2}}{2A}$
$\Rightarrow 2Ax - Dy = \pm \sqrt{(Dy)^2 - 4A^2}$
Squaring both sides and cancelling $D^2y^2$, we are left with
$Ax^2 - Dxy + A = 0$
Sigh! Looks like I'm just going in circles, and while it's easy to prove using the standard results, I can't seem to find the factors. How can this be done?
You almost got it, but you made a couple of careless errors. You should have put $0=Ax^2-Dyx-Ay^2$, and then you forgot that the “constant” term at the right is not $-A$ but $-Ay^2$. So you should have gotten $$ x=\frac{Dy\pm\sqrt{D^2y^2+4A^2y^2}}{2A}=y\frac{D\pm\sqrt{D^2+4A^2}}{2A}\,. $$ Let’s call the two possible constants that multiply $y$ on the right $K_1$ and $K_2$. then the factorizatation is $(x-K_1y)(x-K_2y)$, except that you need a correcting constant in front, I guess it’s $A$.