Show that the orthocentre of triangle formed by the striaght lines $$ax^2 + 2hxy + by^2 = 0$$ and $lx + my = 1$ is point $(x^\prime, y^\prime)$ such that $${x^\prime \over l} = {y^\prime \over m} = {a+ b \over am^2 - 2hlm + bl^2} \tag{+}$$
Let the two lines be $y - \alpha x = 0, y - \beta x = 0$
The altitude from top vertex is $mx - ly = 0 \qquad \qquad (1)$
Now we need to compute the intersection of $y - \alpha x = 0$ and $lx + my - 1 = 0$ which is $\displaystyle \left( {1\over l + m\alpha}, {\alpha \over l + m \alpha} \right)$.
From this, the equation of altitude from intesection of lines $y - \alpha x = 0$ and $lx + my - 1 =0$ can be computed.
I did the labour and got $$\beta \gamma y + \gamma x - (\alpha\beta + 1) = 0\tag 2$$ where $\gamma = l + m \alpha$
The intersection (1) and (2) is the orthocentre which I get as $$(x^\prime, y^\prime) = \left( {m \over \gamma}\left[ {\alpha \beta + 1 \over m + l\beta}\right], { l \over \gamma } \left[ {\alpha \beta + 1 \over m + l\beta} \right]\right) \tag{3}$$
Now from here computing $\displaystyle {x^\prime\over l}$ and $\displaystyle {y^\prime \over m}$
$${y^\prime \over m} ={l\over m} \left({1+ \alpha \beta \over lm + l^2\beta + m^2\alpha + ml\alpha\beta}\right)\tag{4}$$
$${x^\prime \over l} = {m\over l} \left({1+ \alpha \beta \over lm + l^2\beta + m^2\alpha + ml\alpha\beta}\right)\tag{5}$$
We know that $b(y - \alpha)(y - x\beta) := ax^2 + by^2 + 2hxy = 0$
$$\therefore b(y - \alpha)(y - x\beta) = by^2 + byx(-\beta - \alpha) + b\alpha\beta x^2 = 0$$
We get the values of $a,b,h$ in terms of $\alpha, \beta$ from this equation and plugging those in (+) we get $$1+ \alpha \beta \over \alpha\beta m^2 + (\alpha + \beta)(ml) + l^2 \tag{6}$$
From (4), (5) and (6) I get
$${x^\prime \over l} \ne {y^\prime \over m} \ne {a+ b \over am^2 - 2hlm + bl^2}$$
I did this question same way at least twice but still get the same result.
- What went wrong in my answer ?
This is a fairly straightforward, though mildly tedious, calculation if you use homogeneous coordinates. Doing so lets you represent lines in $\mathbb R^2$ as cross products of points and by duality compute their intersection as a cross product. (The vectors that represent lines transform differently than the ones that represent points, but that’s not going to come into play here). You can use the model of the projective plane as the plane $z=1$ in $\mathbb R^3$ to understand how this works: the vector that represents a line is just the normal to the plane defined by the line and the origin.
It turns out that the homogeneous vector that represents the line with equation $ax+by=c$ is simply $a:b:-c$. So, taking two generic lines through the origin, we have for our three lines $$\begin{align}L_0&=l:m:-1\\L_1&=\alpha:\beta:0\\L_2&=\gamma:\delta:0.\end{align}$$ We want $L_1$ and $L_2$ to be distinct, so $\alpha\delta-\beta\gamma\ne0$. For our two points of intersection we then have $$\begin{align}P_1&=L_0\times L_1=-\beta:\alpha:m\alpha-l\beta\\P_2&=L_0\times L_2=-\delta:\gamma:m\gamma-l\delta.\end{align}$$ The point at infinity that is on a line is also its direction vector, so the two altitude lines through these points are $P_1\times L_2$ and $P_2\times L_1$, and thus the orthocenter is $$\begin{align} H&=(P_1\times L_2)\times(P_2\times L_1) \\ &=([-\beta,\alpha,m\alpha-l\beta]\times[\gamma,\delta,0])\times([-\delta,\gamma,m\gamma-l\delta]\times[\alpha,\beta,0]) \\ &=[(m\alpha-l\beta)\delta,-(m\alpha-l\beta)\gamma,\alpha\gamma-\beta\delta]\times[(m\gamma-l\delta)\beta,-(m\gamma-l\delta)\alpha,\alpha\gamma-\beta\delta] \\ &=[l(\alpha\gamma+\beta\delta)(\beta\gamma-\alpha\delta),m(\alpha\gamma+\beta\delta)(\beta\gamma-\alpha\delta),(m\alpha-l\beta)(m\gamma-l\delta)(\beta\gamma-\alpha\delta)] \\ &=l(\alpha\gamma+\beta\delta):m(\alpha\gamma+\beta\delta):(m\alpha-l\beta)(m\gamma-l\delta)\end{align}$$ from which we can read $$\frac{x'}l=\frac{y'}m={\alpha\gamma+\beta\delta\over(m\alpha-l\beta)(m\gamma-l\delta)},$$ which looks pretty close to the desired result. Setting $(\alpha x+\beta y)(\gamma x+\delta y)=ax^2+2hxy+by^2$, we see that the denominator is simply this expression evaluated at $x=m$, $y=-l$, while the numerator is $a+b$, so $$\frac{x'}l=\frac{y'}m={a+b\over am^2-2hlm+bl^2}.$$
Postscript: Although the above computation produces the required result, I find it somewhat unsatisfactory because the last ratio seems to pop up out of nowhere. The left-hand equality has an obvious geometric interpretation: it says that the orthocenter lies somewhere on the line $\lambda(l,m)$, which is the normal to $L_0$ and so also the altitude line through the origin. The geometric meaning of the other equalities escapes me, although I can see that the right-hand fraction is a ratio of dot products of various relevant normals. The trilinear coordinates of a triangle’s orthocenter are $\sec A:\sec B:\sec C$ so perhaps there’s some connection to those ratios.