$ABAB=0_{n\times n}$, then must $BABA$ be $0_{n\times n}$?

Question

Let $A$ and $B$ be $n\times n$ matrices such that $ABAB=0_{n\times n}$. Can we conclude that $BABA$ must be $0_{n\times n}$?

I fail to give a counterexample when $n=2$， so I guess the answer is "yes"

Answer 1

This is Putnam 1990-A5, and there are counterexamples for every $n\ge 3$, i.e., $$ A=\begin{pmatrix} 0 & 0 & 0 \cr 0 & 1 & 0 \cr 0 & 0 & 1\end{pmatrix},\; B=\begin{pmatrix} 0 & 1 & 0 \cr 0 & 0 & 1 \cr 0 & 0 & 0\end{pmatrix}, $$ Then $ABAB=0$, but $BABA=\begin{pmatrix} 0 & 0 & 1 \cr 0 & 0 & 0 \cr 0 & 0 & 0\end{pmatrix}$.

Answer 2

My idea for a counterexample: For $n=3$: take $A$ to be the homomorphism that kills the first line of a vector, $B$ the homomorphism that moves every line one up, but the first line which will stay in place. This can be generalized for $n \geq 3$.

Edit: As I was typing this question was still without answer. Of course the answer above is just the matrix version of what I wrote (with the slight difference that above $B$ actually 'kills' the first line)

Answer 3

For $n = 2$: The characteristic polynomials of $AB$ and $BA$ are the same (for any $n$). Therefore, if $AB$ is nilpotent then $BA$ is also nilpotent. So, we necessarily have $BABA = (BA)^2 = 0$.