On the nilpotence of the matrix $AB-BA$

Question

Given $n\times n$ matrices $A,B$ satisfy:

$rank(AB-BA)=1$

Prove that $(AB-BA)^{2}=0$

Generalize the problem if possible.

Any solution not mention Jordan canonical form would be appreciated!

Answer 1

Hint: show that $\text{tr}(AB - BA) = 0$. Use this fact and the rank to find the Jordan Canonical form of $AB-BA$.

The generalization they want might be "what if $\text{rank}(AB-BA) = r$"?

---

Suppose that $AB - BA = UTU^*$ with $T$ upper triangular. It follows that $T$ has a rank of $1$ and a trace of $0$.

Since $T$ has rank $1$, it has at most one non-zero diagonal entry. However, if $T$ has a single non-zero entry on the diagonal, then it would have a non-zero trace.

So, $T$ is a strictly upper triangular matrix with rank $1$. In particular, $T$ has the form $$ T = \pmatrix{ 0&*&*\\ &\ddots&*\\ &&0 } $$ Note that the rank of $T^2$ must be strictly less than the rank of $T$. It follows that $T^2$ has rank $0$, so that it must be the zero matrix.

Answer 2

Let $e_j, j=1,\cdots,n$ be the canonical basis of $\mathbb{R}^n$. The rank of $M$ is $1$. Hence there exists $a_j\in \mathbb{R}$, not all zero, and a vector $v=(v_1,\cdots,v_n)$, not zero, such that $Me_j=a_jv$ for all $j=1,\cdots,n$. We immediately have that the trace of $M$ is $T=a_1v_1+a_2v_2+\cdots+a_n v_n$. Hence $T=0$.

Now, for $j=1,\cdots,n$:

$$M^2e_j=a_jMv= a_jM(\sum v_k e_k)=a_j(\sum v_k a_k v)=a_jTv=0$$ Hence $M^2=0$.