Linear operator $A$ on a finite -dimensional vector space

Question

Let $A$ and $B$ be linear operators on a finite dimensional vector space $V$ over $\mathbb{R}$ such that $AB=(AB)^2$. If $BA$ is invertible then which of the following is true:

(a) $BA = AB$ on $V$
(b) $\operatorname{tr}(A)$ is non zero
(c) $0$ is an eigenvalue of $B$
(d) $1$ is an eigenvalue of $A$

I found that (b) is true but how to show that all other are false?

Answer 1

Hint: If $BA$ is invertible, then $A,B,$ and $AB$ must all be invertible as well.

In fact, of all these statements, only (a) is necessarily true.

Consider the matrices $$ A = \pmatrix{2&0\\0&-2}, \quad B = \pmatrix{1/2 & 0\\0&-1/2} $$ This counterexample shows that every statement except (a) is false.

---

Hint: as the other answer notes, we can deduce that $AB=I$. Use this to deduce that $AB=BA$. We assume here that both $A$ and $B$ are operators from $V$ to $V$.

Answer 2

If $BA$ is invertible then so are $A,B$. Since $AB=ABAB$, multiplying on the left by $A^{-1}$ and on the right by $B^{-1}$ gives $I = BA$.