Does $\text{rank}(AB)= \text{rank}(BA)= \text{rank}(A)$ imply that B is invertible?

Question

(After revision) I would like to prove that if $A$ and $B$ are nonzero, $\text{rank}(AB)= \text{rank}(BA)= \text{rank}(A)$ imply that B is invertible, but don’t know where to start the proof. Any help or hints?

Answer 1

$\DeclareMathOperator\rank{rank}$The equivalence you might be looking for is:

An $n\times n$ matrix $B$ is invertible if and only if $\rank(AB)=\rank(BA)=\rank(A)$ holds for all $n\times n$ matrices $A$.

The proof of the first direction was your original homework question:

If $B$ is an invertible $n\times n$ matrix then $\rank(AB)=\rank(BA)=\rank(A)$ holds for all $n\times n$ matrices $A$.

The other direction is:

If $B$ is an $n\times n$ matrix such that $\rank(AB)=\rank(BA)=\rank(A)$ holds for all $n\times n$ matrices $A$, then $B$ is invertible.

To prove this, note that since the equation holds for all $A$ by assumption, it also holds for $A=I$ (the identity matrix), which yields $$ \rank(B) = \rank(I) = n. $$ Hence $B$ is of full rank and invertible.

---

However, in your question you proposed the following statement

If $A$ and $B$ are non-zero $n\times n$ matrices and $\rank(AB)=\rank(BA)=\rank(A)$ holds, then $B$ is invertible.

This statement is false as shown by the counter example in morphy22's answer.

One counter example is enough to show that this statement is false in general and hence it is impossible to prove it.

Answer 2

I have a counterexample.
Try : $$A=B= \begin{bmatrix} 1&0\\0&0\end{bmatrix}$$