Let $ A, B \in M_n (\mathbb{C})$ such that $(A-B)^2 = A -B$. Then $\mathrm{rank}(A^2 - B^2) \geq \mathrm{rank}( AB -BA)$.
I tried to apply the basic inequalities without results. How to start? Thank you.
Let $ A, B \in M_n (\mathbb{C})$ such that $(A-B)^2 = A -B$. Then $\mathrm{rank}(A^2 - B^2) \geq \mathrm{rank}( AB -BA)$.
I tried to apply the basic inequalities without results. How to start? Thank you.
On
$\newcommand{\rank}{\mathrm{rank}}$ $\newcommand{\diag}{\mathrm{diag}}$ $\newcommand{\M}{\begin{pmatrix} I_{(r)} & 0 \\ 0 & 0 \end{pmatrix}}$ The key to this problem is to use the property of idempotent matrix $A - B$. As @Berci suggested, denote $A - B = P$, then $P^2 = P$, which implies there exists an order $n$ invertible matrix $T$, such that $P = T\diag(I_{(r)}, 0)T^{-1}$, where $r = \rank(P)$ (this is a standard result for idempotent matrix, if you need a proof for that, I can add it as appendix).
Now $A = B + P$, substituting it to $A^2 - B^2$ and $AB - BA$ respectively, we have $A^2 - B^2 = P + BP + PB, AB - BA = PB - BP$. Equivalently, we have \begin{align*} & T^{-1}(A^2 - B^2)T = \M + T^{-1}BT\M + \M T^{-1}BT, \tag{1} \\ & T^{-1}(AB - BA)T = \M T^{-1}BT - T^{-1}BT\M. \tag{2} \end{align*}
Partition $T^{-1}BT$ as $\begin{pmatrix} A_{11} & A_{12} \\ A_{21} & A_{22} \end{pmatrix}$, where $A_{11}$ is of order $r$, then $(1)$ and $(2)$ become \begin{align*} & T^{-1}(A^2 - B^2)T = \begin{pmatrix} I_{(r)} + 2A_{11} & A_{12} \\ A_{21} & 0 \end{pmatrix}, \\ & T^{-1}(AB - BA)T = \begin{pmatrix} 0 & A_{12} \\ -A_{21} & 0 \end{pmatrix}. \end{align*}
In view of this, to show that $\rank(A^2 - B^2) \geq \rank(AB - BA)$, it suffices to show \begin{align*} \rank\begin{pmatrix} B_1 & B_2 \\ B_3 & 0\end{pmatrix} \geq \rank\begin{pmatrix} 0 & B_2 \\ -B_3 & 0\end{pmatrix} \end{align*} for any order $r$ matrix $B_1$, $r \times (n - r)$ matrix $B_2$, and $(n - r) \times r$ matrix $B_3$. To this end, let $\begin{pmatrix} 0 \\ e_1 \end{pmatrix}, \ldots, \begin{pmatrix} 0 \\ e_i \end{pmatrix}, \begin{pmatrix} e_{i + 1} \\ 0 \end{pmatrix}, \ldots, \begin{pmatrix} e_{i + j} \\ 0 \end{pmatrix}$ be the maximal linear independent group of $\begin{pmatrix} 0 & B_2 \\ -B_3 & 0\end{pmatrix}$, where $e_1, \ldots, e_i$ are columns of $B_3$, and $e_{i + 1}, \ldots, e_{i + j}$ are columns of $B_2$, $0 \leq i \leq r$, $0 \leq j \leq n - r$. It can then be shown that columns $\begin{pmatrix} f_1 \\ e_1 \end{pmatrix}, \ldots, \begin{pmatrix} f_i \\ e_i \end{pmatrix}, \begin{pmatrix} e_{i + 1} \\ 0 \end{pmatrix}, \ldots, \begin{pmatrix} e_{i + j} \\ 0 \end{pmatrix}$ are linearly independent, where $f_1, \ldots, f_i$ are corresponding columns of matrix $B_1$. To wit, suppose \begin{align*} a_1\begin{pmatrix} f_1 \\ e_1 \end{pmatrix} + \cdots + a_i\begin{pmatrix} f_i \\ e_i \end{pmatrix} + a_{i + 1}\begin{pmatrix} e_{i + 1} \\ 0 \end{pmatrix} + \cdots + a_{i + j}\begin{pmatrix} e_{i + j} \\ 0 \end{pmatrix} = 0, \end{align*} then \begin{align*} & a_1f_1 + \cdots + a_if_i + a_{i + 1}e_{i + 1} + a_{i + j}e_{i + j} = 0 \tag{3} \\ & a_1e_1 + \cdots + a_ie_i = 0. \tag{4} \end{align*} By linear independence of $e_1, \ldots, e_i$, from $(4)$ we obtain $a_1 = \cdots = a_i = 0$. Substituting these back to $(3)$, we have $a_{i + 1}e_{i + 1} + \cdots + a_{i + j}e_{i + j} = 0$, which implies $a_{i + 1} = \cdots = a_{i + j} = 0$ by linear independence of $e_{i + 1}, \ldots, e_{i + j}$. This shows $\begin{pmatrix} f_1 \\ e_1 \end{pmatrix}, \ldots, \begin{pmatrix} f_i \\ e_i \end{pmatrix}, \begin{pmatrix} e_{i + 1} \\ 0 \end{pmatrix}, \ldots, \begin{pmatrix} e_{i + j} \\ 0 \end{pmatrix}$ are linearly independent, whence \begin{align*} \rank\begin{pmatrix} B_1 & B_2 \\ B_3 & 0\end{pmatrix} \geq i + j = \rank\begin{pmatrix} 0 & B_2 \\ -B_3 & 0\end{pmatrix}. \end{align*}
This completes the proof.
Hint: Let $P:=A-B$, then $A=P+B$, $P^2=P$ and $$A^2-B^2= (P+B)^2-B^2=P^2+PB+BP=P+PB+BP\\ AB-BA=(P+B)B-B(P+B)=PB-BP$$ Now we can apply a basis transformation using an eigenbasis of $P$ so that the matrix of $P$ becomes diagonal with $k:={\rm rank}(P)$ ones and $n-k$ zeroes, so $PB$ gives the first $k$ columns of $B$ then $n-k$ zero columns, and similarly $BP$ gives the first $k$ rows of $B$ then $n-k$ zero rows.