Steps taken:
- Multiply both sides by $A^{-1}$, given $A$ is invertible.
$A^{-1}AB - A^{-1}BA = A^{-1}A $
Thus, $B - A^{-1}BA = I$
- Take the trace of both sides.
$\operatorname{Tr}(B-A^{-1}BA) = \operatorname{Tr}(I)$
- Utilize the linear mapping property of trace.
$\operatorname{Tr}(B)-\operatorname{tr}(A^{-1}BA) = \operatorname{Tr}(I)$
- Utilize the cyclic property of trace.
$\operatorname{Tr}(B)-\operatorname{tr}(BAA^{-1}) = \operatorname{Tr}(I)$
$\operatorname{Tr}(B)-\operatorname{tr}(BI) = \operatorname{Tr}(I)$
$\operatorname{Tr}(B)-\operatorname{tr}(B) = \operatorname{Tr}(I)$
- Reaching a contradiction
$0 = \operatorname{Tr}(I)$
Is my usage of the trace properties correct?
It is correct.
Here is a different, more high level proof. Consider the linear map $\delta : M_n \to M_n$ given by $$\delta(X):= XB-BX.$$
Then we can show by induction that for all $k \in \Bbb{N}$ holds $\delta(A^k) = kA^k$. Indeed, for $k=1$ the claim is true by assumption. Assuming it holds for some $k$, we have \begin{align} \delta(A^{k+1}) &= A^{k+1}B-BA^{k+1} \\ &= A^k(AB-BA) + (A^kB - BA^k)A \\ &= A^k\delta(A) + \delta(A^k)A \\ &= (k+1)A^{k+1} \end{align}
so it holds for $k+1$ as well.
If all $A^k$ were nonzero, we would get that $k$ is an eigenvalue of $\delta$ for all $k \in \Bbb{N}$. This is impossible since there are at most $n^2$ different eigenvalues of $\delta$ so it has to be $A^k =0$ for some $k \in \Bbb{N}$. This is a contradiction since $A$ is invertible.