Are block-transposes of matrices with commuting blocks similar?

Question

Suppose $A$ is an $nm \times nm$ block matrix over some field, with $n^2$ blocks each of size $m \times m$. Suppose further that the blocks commute. Let $A^{BT}$ denote the "block transpose" of $A$, namely the $nm \times nm$ block matrix whose $(j,i)$ block is the $(i,j)$ block of $A$. That is, the positions of the blocks have been transposed, but not the blocks themselves.

Is $A$ similar to $A^{BT}$?

If $m=1$, the answer is yes. I can show that $A$ and $A^{BT}$ have the same minimal and characteristic polynomials in general.

For example, suppose $A, B, C, D$ are commuting $2 \times 2$ matrices. Is it necessarily the case that \begin{align*} \begin{pmatrix} A & B \\ C & D \end{pmatrix} \sim \begin{pmatrix} A & C \\ B & D \end{pmatrix}? \end{align*}

Answer 1

When $m\ge4$, the answer is "no". Consider $m=2k$ and $A,B,C,D\in M_{2k}(\mathbb F)$ such that $$ A=\pmatrix{0&X\\ 0&0}, \ B=\pmatrix{0&Y\\ 0&0}, \ C=\pmatrix{0&Z\\ 0&0}, \ D=\pmatrix{0&W\\ 0&0} $$ for some $X,Y,Z,W\in M_k(\mathbb F)$. Clearly $A,B,C,D$ commute, because the product of any two of them is zero. However, when $k\ge2$, we have $$ \text{rank of }\pmatrix{A&B\\ C&D}=\text{rank of }\pmatrix{X&Y\\ Z&W}\ne\text{rank of }\pmatrix{X&Z\\ Y&W}=\text{rank of }\pmatrix{A&C\\ B&D} $$ in general. Therefore the matrix on the left is not always similar to its blockwise transpose. E.g. let $k=2$ and $\{X,Z,Y,W\}$ be the standard basis of $M_k(\mathbb F)$. Then $$ \text{rank of }\left(\begin{array}{cc|cc}1&0&0&1\\ 0&0&0&0\\ \hline0&0&0&0\\ 1&0&0&1\end{array}\right) =2\ne4= \text{rank of }\left(\begin{array}{cc|cc}1&0&0&0\\ 0&0&1&0\\ \hline0&1&0&0\\ 0&0&0&1\end{array}\right). $$

Answer 2

In response to the comments on the question: the statement will hold if $A,B,C,D$ are diagonalizable.

Suppose that $A,B,C,D$ are diagoanlizable. Because they commute, they are simultaneously diagonalizable, which is to say that there exists an invertible $S$ such that $$ \Lambda_A = S^{-1}AS, \quad \Lambda_B = S^{-1}BS, \quad \Lambda_C = S^{-1}CS, \quad \Lambda_D = S^{-1}DS $$ are all diagonal. We note that $$ \pmatrix{S \\ & S}^{-1} \pmatrix{A&B\\C&D}\pmatrix{S\\&S} = \pmatrix{\Lambda_A & \Lambda_B \\ \Lambda_C & \Lambda_D}. $$ On the other hand, this matrix is similar to its transpose. Putting these together gives us that $$ \pmatrix{A&B\\C&D} \sim \pmatrix{\Lambda_A & \Lambda_C\\ \Lambda_B & \Lambda_D}. $$ Finally, we have $$ \pmatrix{S \\ & S} \pmatrix{\Lambda_A&\Lambda_C\\\Lambda_B&\Lambda_D}\pmatrix{S\\&S}^{-1} = \pmatrix{A & C\\ B& D}, $$ which means that we have $$ \pmatrix{A & B\\C&D} \sim \pmatrix{A & C\\B&D}, $$ which was what we wanted.