Question:
Let $A,B$ be $n\times n$ complex matrices, and let $a$ and $b$ be complex numbers such that $$AB-BA=aA+bB.$$ Show that $A$ and $B$ are simultaneously diagonalizable.
My Try:
Case 1: If $a=b=0$, then $AB=BA$. This is true. [Editor's note: In this case, the result is well known.]
Case 2: Without loss of generality, we let $a\neq 0$, so we may assume $a=1$ (let $\dfrac{1}{a}B$ replace $B$).
Let $C=AB-BA=A+bB$, then $$CB-BC=C$$ and I can't continue.
Thank you very much!
I've got a partial answer assuming that $A$ is diagonalizable and the nonzero $b$ is not equal to a difference of any pair of the distinct eigenvalues of $A$. Hope it will help :-)
So let $AB-BA=aA+bB$, $b\neq 0$, and $A$ be diagonalizable. Then there exists a nonsingular $X$ such that $A=X\Lambda X^{-1}$. We can assume that the matrix $A\in\mathbb{C}^{n\times n}$ has $k$ distinct eigenvalues $\lambda_1,\ldots,\lambda_k$ such that $$ \Lambda = \begin{bmatrix}\lambda_1 I_{n_1} & & \\ & \ddots & \\ & & \lambda_k I_{n_k}\end{bmatrix}, $$ where $I_{n_i}$ is the identity matrix of the dimension $n_i$ and $\sum_{i=1}^k n_i=n$. From $AB-BA=aA-bB$, we have $$ \Lambda\tilde{B}-\tilde{B}\Lambda=a\Lambda+b\tilde{B},\tag{1} $$ where $\tilde{B}=X^{-1}BX$. Assume $\tilde{B}$ to be partitioned to blocks conforming to the partitioning of $\Lambda$: $$ \tilde{B} = \begin{bmatrix}\tilde{B}_{11} & \cdots & \tilde{B}_{1k} \\ \vdots & \ddots & \vdots \\ \tilde{B}_{k1} & \cdots & \tilde{B}_{kk}\end{bmatrix}, $$ where $\tilde{B}_{ij}\in\mathbb{C}^{n_i\times n_j}$. Now compare both sides of (1) for each block:
Note that (2) at least implies that for a fixed pair of $i\neq j$, at least one of the blocks $\tilde{B}_{ij}$ and $\tilde{B}_{ji}$ is necessarily zero. I don't know now how to show that they are both zero, but maybe the assumption that $B$ is diagonalizable as well could help.
Also, the assumption that both $A$ and $B$ are diagonalizable seems reasonable. In fact, it is also required to prove that if $A$ and $B$ commute then $A$ and $B$ are simultaneously diagonalizable.