If $A$ commutes with $(AB - BA)^2$, is $\det(AB - BA) = 0$?

Question

We have $A$ and $B$ are $3 \times 3$ matrices with complex numbers. We know matrix $A$ is commuting with matrix $(AB-BA)^2$. Can you show $\det(AB-BA)=0$?

I tried using some Hamilton Cayley Theorem on matrix $AB-BA$ but I got nothing. I tried using some characteristic polynomial on $A \times (AB-BA)^2$ but I got nothing. We should use eigenvalues? I don't know where do I need to use the fact that $A$ and $(AB-BA)^2$ are commuting.

Answer 1

Yes. Since $C=[A,B]=AB-BA$ is traceless, its characteristic polynomial is of the form $p(x)=x^3+ax-\det(C)$. So, by Cayley-Hamilton theorem, $$ C^4+aC^2-\det(C)C=Cp(C)=0.\tag{1} $$ Since $A$ commutes with $C^2$, $(1)$ implies that $A$ commutes with $\det(C)C=[A,\det(C)B]$. Therefore $\det(C)C$ is nilpotent, by Jacobson’s lemma. Hence $C$ is singular.