Suppose that for two $n \times n$ matrices $A,B$ we have $AB = A + B$. Prove that $\text{rank}(A^2) + \text{rank} (B^2) \leq 2 \text{rank} (AB).$

Question

Suppose that for two $n \times n$ matrices $A,B$ we have $AB = A + B$. Prove that $\text{rank}(A^2) + \text{rank} (B^2) \leq 2 \text{rank} (AB).$

This reminds me of Sylvester's Rank Inequality theorem, but I'm not sure if that's really helpful here. I haven't really made significant progress on this beyond writing out a few matrix multiplication. Would appreciate any help at all! Thank you.

Answer 1

Step 1: $B-I$ is invertible

In fact, $$ Bv = v \implies Av = ABv = Av + Bv = Av + v\implies v=0. $$

Step 2: $AB=BA$

In fact, since $B-I$ is invertible, in particular $(B-I)^{-1}=P(B)$ where $P$ is a polynomial (obtained by Euclidean division between the characteristic polynomial and $x-1$). So $$ AB = A+B \implies A(B-I) = B \implies A = BP(B) = P(B)B $$ and $A$ is thus a polynomial in $B$. In particular $AB=BA$.

Step 3: Profit

Notice now that $A^2 = P(B)BA$, so $rk(A^2)\le rk(BA)$ and the problem is symmetric in $B,A$, so $rk(B^2)\le rk(AB)$, leading to $$ rk(A^2) + rk(B^2)\le rk(BA)+ rk(AB) = 2rk(AB) $$

Answer 2

There's a generalisation of Sylvester's Rank Inequality Theorem attributed to Frobenius. It states for all matrices $X, Y, Z$ we have $$ rk(XY) + rk(YZ) \le rk(Y) + rk(XYZ). $$ Using $AB = A + B$ we get $(A-I)(B-I)=I$ hence $(A-I) = (B-I)^{-1}$ and so $(B-I)(A-I) = I$ which implies $BA = A + B = AB$. So the matrices commute. Then apply the above theorem with $Y = AB$, $X = A-I$ and $Z = B-I$ gives the desired inequality.

Answer 3

The condition $AB=A+B$ is equivalent to $XY=I$ where $X=A-I$ and $Y=B-I$. It follows that \begin{align*} &2\operatorname{rank}(AB)-\operatorname{rank}(A^2)-\operatorname{rank}(B^2)\\ &=2\operatorname{rank}(X+Y+2I)-\operatorname{rank}(X^2+2X+I)-\operatorname{rank}(Y^2+2Y+I)\\ &=2\operatorname{rank}\big(X(X+Y+2I)\big)-\operatorname{rank}(X^2+2X+I)-\operatorname{rank}\big(X^2(Y^2+2Y+I)\big)\\ &=2\operatorname{rank}(X^2+I+2X)-\operatorname{rank}(X^2+2X+I)-\operatorname{rank}(I+2X+X^2)\\ &=0. \end{align*} Hence the inequality in question holds and it is actually an equality.