Find max of $S = x\sin^2\angle A + y\sin^2\angle B + z\sin^2\angle C$

Question

Let $x$, $y$, $z$ are positive constants. $A$, $B$, $C$ are three angles of the triangle. Prove that $$S = x \sin^2 A + y \sin^2 B + z \sin^2 C \leq \dfrac{\left(yz+zx+xy\right)^2}{4xyz}$$ and find when it holds equality

Answer 1

Update

This is an intricate problem, since not only the value of $S_{\max}$ is changing with the values of the parameters $x$, $y$, $z$, but also the global extremal situation.

Claim: If $0<2x<y\leq z$ then $S_{\max}=y+z$.

Proof: If one of the angles is $>{\pi\over2}$, e.g., $\alpha\leq\beta<{\pi\over2}<\gamma$, it is possible to enlarge $S$ by replacing $\alpha$, $\beta$,$\gamma$ with $$\alpha':=\alpha+\gamma-{\pi\over2}, \quad \beta':=\beta,\quad \gamma':={\pi\over2}\ .$$ It follows that we may assume $0\leq\alpha\leq\beta\leq\gamma\leq{\pi\over2}$, in this order. Write $\beta:={\pi\over2}-\beta'$, $\>\gamma:={\pi\over2}-\gamma'$, $\alpha:=\beta'+\gamma'$. Then $S$ appears as $$S=\sin^2(\beta'+\gamma') x+(1-\sin^2\beta')y+(1-\sin^2\gamma')z\ .$$ Now by Schwarz' inequality $$\sin(\beta'+\gamma')=\cos\gamma'\sin\beta'+\cos\beta'\sin\gamma'\leq\sqrt{\cos^2\gamma'+\cos^2\beta'} \sqrt{\sin^2\beta'+\sin^2\gamma'}\ ,$$ so that $$\sin^2(\beta'+\gamma')\leq 2(\sin^2\beta'+\sin^2\gamma')\ .$$ It follows that $$S\leq y+z-(y-2x)\sin^2\beta'-(z-2x)\sin^2\gamma'\leq y+z$$ with equality iff $\beta'=\gamma'=0$.$\quad\square$

On the other hand, if $x=y=z=1$, then $\alpha=0$, $\beta=\gamma={\pi\over2}$ leads to $S=2$, whereas $\alpha=\beta=\gamma={\pi\over3}$ produce $S={9\over4}$. This shows that there is a "phase transition" where the morphology of the extremal situation changes.