Let $A+B+C=\pi$ and $\sin^2(A)+\sin^2(B)-\sin^2(C)=p$, then find range of $p$
Approach: After simplifying above expression I obatained $p=2\sin A \sin B \cos C$.
As I can see maximum value of above expression is $2$ with guess of $A=\dfrac{\pi}{2}, B=\dfrac{\pi}{2}, C=0$.
But I am not able to find its minimum value. Also is there proper method to solve this problem other than gueesing.
We observe that $\sin(A), \sin(B)$ are positive but $\cos(C)$ can be negative, so, the minimum must be less than $0$ for $C\ge \pi/2$. So, we seeks the minimum value under the constraints $A+B+C = \pi$ and $C \ge \pi/2$, we have: $$\sin(A)\sin(B) \le \left(\frac{\sin(A)+\sin(B)}{2}\right)^2=\sin^2\underbrace{\left(\frac{A+B}{2}\right)}_{=\frac{\pi}{2}-\frac{C}{2}}\cos^2\left(\frac{A-B}{2}\right)\le \cos^2\left( \frac{C}{2}\right)$$ $$\implies p\ge2 \cos^2\left( \frac{C}{2}\right)\cos(C)=-2 \cos^2\left( \frac{C}{2}\right) \left(1-2 \cos^2\left( \frac{C}{2}\right)\right)$$ with $\cos\left( \frac{C}{2}\right) \in \left[\frac{1}{\sqrt{2}}, 1\right]$ (as $\pi/2 \le C\le \pi$)
Applying the AM-GM inequality: $$p \ge -\left(\frac{2 \cos^2\left( \frac{C}{2}\right)+\left(1-2 \cos^2\left( \frac{C}{2}\right)\right)}{2}\right)^2=-\frac{1}{4}$$ The equality occurs if and only $(A,B,C) =(\pi/6,\pi/6,2\pi/3)$.