Trace inequality for $\operatorname{Tr}(ABAB)$?

Question

Is it true that for square matrices $A,B$ it holds that $$\operatorname{Tr}(A^2)\operatorname{Tr}(ABAB)\le (\operatorname{Tr}(A^2 B))^2$$ maybe under some additional assumptions on $A,B$ like positive semidefinite?

Background: In a more general setting I obtained $$\operatorname{Tr}(A\bar A)\operatorname{Tr}(AB\bar A \bar B)+\lvert\operatorname{Tr}(AB\bar A)\rvert^2+\cdots\ge 0$$ for complex-symmetric $A$ and general $B$, where $\bar A$ denotes the entry-wise complex conjugate. I am wondering whether the sum of the two terms above is always non-negative on their own without the rest of the sum. But then I noticed that even in the setting of real-valued matrices I do not know whether such an inequality might hold true.

Answer 1

Here is a counterexample, namely $$ A=\begin{pmatrix} 0 & 1 & 0 \cr 1 & 0 & 0 \cr 1 & 0 & 1 \end{pmatrix}, \; B=\begin{pmatrix} 0 & 0 & 1 \cr 1 & 1 & 0 \cr 1 & 0 & 0 \end{pmatrix}. $$ Then we have $$ \operatorname{Tr}(A^2)\operatorname{Tr}(ABAB)=6,\; (\operatorname{Tr}(A^2 B))^2=4. $$ Also the converse inequality does not hold in general. Take $$ A=\begin{pmatrix} 0 & 1 & 0 \cr 1 & 0 & 0 \cr 0 & 0 & 1 \end{pmatrix}, \; B=\begin{pmatrix} 0 & 0 & 1 \cr 0 & 1 & 0 \cr 1 & 0 & 0 \end{pmatrix}. $$ Then $$ \operatorname{Tr}(A^2)\operatorname{Tr}(ABAB)=0,\; (\operatorname{Tr}(A^2 B))^2=3. $$

Answer 2

If $A$ is normal then the polar decomposition of $A$ can be written $A = U\,|A|$ with $U$ unitary and commuting with $A$ (see e.g. here) and so $$ |\mathrm{Tr}(A^2B)|^2 = \left|\mathrm{Tr}\left(\sqrt{|A|}\,B\,\sqrt{|A|}\,U\,A\right)\right|^2 \\ \leq \mathrm{Tr}\left(\left|\sqrt{|A|}\,B\,\sqrt{|A|}\right|^2\right) \,\mathrm{Tr}(|A|^2) \\ \leq \mathrm{Tr}(|A|\,B\,|A|\,B) \,\mathrm{Tr}(|A|^2) $$ so in particular if $A$ is positive semi-definite, we get the reverse to your inequality $$ |\mathrm{Tr}(A^2B)|^2 \leq \mathrm{Tr}(A\,B\,A\,B) \,\mathrm{Tr}(A^2). $$