When $a$, $b$, and $c$ are unit vectors, what is the minimum value of $(a\cdot b)(b\cdot c)(c\cdot a)$?
If the three vectors are coplanar and mutually separated by $120^\circ$, then we get a value of $(-\frac12)^3=-\frac18$. Replacing any vector with its negative produces another configuration with the same value. I conjecture that this is the minimum. How would I prove this?
Equivalently, I want to prove that: $$\|a\|^2\|b\|^2\|c\|^2+8(a\cdot b)(b\cdot c)(c\cdot a)\ge0$$for all (not necessarily unit) vectors, with equality iff the three vectors are coplanar and the pairwise angles are each either $120^\circ$ or $60^\circ$.
One obvious approach is to use Cauchy–Schwartz inequality, $-{\|a\|}{\|b\|}\le a\cdot b\le{\|a\|}{\|b\|}$. However, we only get equality on that lower bound when $a$ and $b$ are antiparallel.
Applying Cauchy–Schwartz to each of the three inequalities, we obtain: $$-\|a\|^2\|b\|^2\|c\|^2\le(a\cdot b)(b\cdot c)(c\cdot a)$$ For unit vectors, it tells us that we'd get a minimum value of $-1$ if the three vectors were mutually antiparallel. However, this is an impossible configuration. So Cauchy–Schwartz looks useless.
Is there a (hopefully elementary) way to prove the above inequality, and confirm that, when $\|a\|=\|b\|=\|c\|=1$, the minimum value of $(a\cdot b)(b\cdot c)(c\cdot a)$ is $\frac18$?
HINT
Indicating with $0\le\alpha,\beta,\gamma\le \pi$ the angles between each pair of the three vectors, the condition is equivalent to minimize
under the constraint
By Lagrange's multipliers we find
which implies that the minimum is reached when
thus the extreme value is