If x-y axes are inclined at angle $\omega$ then show that equation $$x^2 + 2xy\cos \omega + y^2 \cos 2\omega = 0$$ represents a pair of perpendicular straight lines passing through origin.
Let two lines with inclination to axis x as $m, n$.
Then the condition of perpendicularity of these lines is $mn + 1 + (m + n)cos \omega = 0$
After some manipulation we get,
$$m = -{n\cos\omega + 1\over \cos \omega + n}$$
So the lines are $$y - nx\ \text{and} \ y + \left({n\cos\omega + 1\over \cos \omega + n}\right)x $$
So pair of straight lines would be
$$(y - nx) \times \left(y + \left({n\cos\omega + 1\over \cos \omega + n}\right)x\right) = 0$$
On Expanding,
$$y^2(\cos \omega - n) - xyn(\cos \omega + n) + xy(n\cos \omega + 1) + x^2(-n)(n\cos\omega + 1) = 0$$
On simplification,
$$y(\cos \omega - n) + xy(1 - n^2) + x^2(-n^2\cos \omega - n) = 0$$
How do I remove $n$ from this equation and get the "to be proven" equation ?
You’ve stated a condition for an arbitrary pair of lines to be orthogonal in the given coordinate system, so it’s not terribly surprising that there’s a left-over free variable. In this problem, though, you’re dealing with a specific pair of lines, so you need to compute their slopes and use those values for $m$ and $n$. The equation can be factored into $$(x+y\cos\omega)^2-(y\sin\omega)^2 = (x+y(\cos\omega+\sin\omega))(x+y(\cos\omega-\sin\omega))=0$$ from which the slopes of the two lines are $-(\cos\omega+\sin\omega)^{-1}$ and $-(\cos\omega-\sin\omega)^{-1}$. Substituting these values into your orthogonality condition eventually yields $0$ as required.
That said, working with slopes might not be the best approach since doing so omits the case of a line parallel to the $y$-axis, which you’ll then have to treat separately for a complete proof. I would solve this problem by transforming the equation to an orthogonal coordinate system. The mapping from a standard Cartesian coordinate system to the one in the problem is $$\begin{align}x&=x'-y'\cot\omega\\y&=y'\csc\omega.\end{align}$$ Substituting this into the equation of the lines and simplifying yields $x'^2-y'^2=0$, which can be seen to be a pair of perpendicular lines by factoring this equation.