If $ax^2+2hxy+by^2=0$ be the two sides of a parallelogram and $px+qy=1$ is one diagonal then prove that the other diagonal is $y(bp-hq)=x(aq-hp)$.
My solution is here; $ax^2+2hxy+by^2=0$ Multiplying by a and adding h^2y^2, $(ax+hy)^2=y^2(h^2-ab) ax+hy=+/- y√(h^2-ab)$
so, $ax+y√(h^2-ab) + hy=0, ax-y√(h^2-ab) + hy=0$ are the two straight lines represented by the given equation and also O(0,0) is the point of intersection of these lines. Now, How do I move further?
You have got the two lines $L_1$ and $L_2$:
$$x = \dfrac{-h\pm\sqrt{h^2-ab}}{a}y.$$
Next step is to find the points of intersection $(x_1,y_1)$ and $(x_2,y_2)$ of these lines with the known diagonal $D_1$ with equation $\;px+qy=1$.
Finding $(x_1,y_1):\quad$ Substitute the equation for $D_1$ into the equation for $L_1$ to give:
$$ax_1 = \left(-h+\sqrt{h^2-ab}\right)\left(1-px_1\right)/q.$$
Simplify to:
$$x_1 = \dfrac{-h+\sqrt{h^2-ab}}{aq-ph+p\sqrt{h^2-ab}},\qquad\text{and then } \qquad y_1 = \dfrac{1-px_1}{q}.$$
Finding $(x_2,y_2):\quad$ Substitute the equation for $D_1$ into the equation for $L_2$ to give:
$$ax_2 = \left(-h-\sqrt{h^2-ab}\right)\left(1-px_2\right)/q.$$
Simplify to:
$$x_2 = \dfrac{-h-\sqrt{h^2-ab}}{aq-ph-p\sqrt{h^2-ab}},\qquad\text{and then } \qquad y_2 = \dfrac{1-px_2}{q}.$$
$$\\$$
Now, the unknown diagonal passes through the origin and also the point $(x_1+x_2,\;y_1+y_2)$. So it has equation:
\begin{align} y &= \dfrac{y_1+y_2}{x_1+x_2} x \\ & \\ &= \left(\dfrac{\dfrac{1-px_1}{q} + \dfrac{1-px_2}{q}}{x_1+x_2}\right) x \\ & \\ &= \left(\dfrac{2}{q(x_1+x_2)} - \dfrac{p}{q}\right) x \\ & \\ &= \dfrac{aq-hp}{bp-hq} x \qquad\text{after substituting for $x_1,\; x_2$ and some tedious simplification.} \end{align}