Find the extremum value of $f(x,y)=ax^2+2hxy+by^2$ subject to constraints $g(x,y)=x^2+y^2-c^2=0$, where $abc \neq 0$ and $(a+1)^2+4(a^2-b^2) \geq 0$.
My attempt: I have use Lagrange Multipliers to solve this. Let us assume $$\phi(x,y, \lambda)=ax^2+2hxy+by^2+ \lambda (x^2+y^2-c^2).$$ Then $$\triangle _{x,y,\lambda}\phi =(2ax+2hy+2\lambda x,\> 2by+2hx+2 \lambda y,\> x^2+y^2-c^2).$$ Hence $$\triangle _{x,y,\lambda}\phi =0$$ implies \begin{align} (a+\lambda)x+hy & =0\\ hx+(b+\lambda)y &=0\\ x^2+y^2-c^2 & =0. \end{align} Then $$x=\frac{-(b+\lambda)y}{h} \implies x \neq 0, \> y \neq 0 \text{ as } abc \neq 0.$$ Now \begin{align} &(a+\lambda)x+hy =0\\ \implies & (a+ \lambda)(b+\lambda)y=h^2y \\ \implies & y=0 \text{ or } {\lambda}^2+(a+b)\lambda +(ab-h^2)=0. \end{align} Since $y \neq 0$ we have ${\lambda}^2+(a+b)\lambda +(ab-h^2)=0.$ Then \begin{align}\lambda & =\frac{-(a+b) \pm \sqrt{(a+b)^2-4ab+4h^2}}{2} \\ &= \frac{-(a+b) \pm \sqrt{(a-b)^2+4h^2}}{2}\end{align} But I cannot proceed further. PLease help me to find the extremum value of $f$.
An alternative approach is using polar coordinates. WLOG, let $c=1$. Then you need to find the extrema of
$$a\cos^2t+2h\cos t\sin t+b\sin^2t=\frac{a-b}2\cos2t+\frac{a+b}2+h\sin2t.$$
As this expression describes a sinusoid, the values of the extrema are immediate:
$$\frac{a+b\pm\sqrt{(a-b)^2+4h^2}}2.$$
(If $c\ne1$, times $c^2$.)