How to solve $C^T = A^{-1} C B^T$ for matrix $C$

Question

I have $2$ known circulant square matrices $A$ and $B$. Then I have an unknown circulant square matrix $C$ that I wish to solve for. I also know that $A C^T = X$ and $B C^T = Y$. For $X$ and $Y$ I only know a condition that should be maintained. That is, $X = C B^T$ and $Y = C A^T$. My first step now was to equate $X=A C^T = C B^T$ and $Y=B C^T = C A^T$, but then $C^T$ would be $C^T = A^{-1} C B^T$ and I am not able to proceed. Is there a way to extract the matrix $C$ from the given conditions?

All matrices are circulant square matrices and thus invertible. You would do me a great favour with your answers. Please feel free to write your comments. Thanks!

Answer 1

Ignoring for the moment, the circulant nature of the matrices, vectorization can be used to find a solution
$$\eqalign{ BC^T &= CA^T \cr (I\otimes B)K\,c &= (A\otimes I)\,c \cr (A^{-1}\otimes I)(I\otimes B)K\,c &= c \cr (A^{-1}\otimes B)K\,c &= c \cr Mc &= c \cr }$$ where $c={\rm vec}(C)\,$ and $\,K$ is the Kronecker commutation matrix. Evidently, $c$ is an eigenvector of $M$ whose associated eigenvalue is $1$.

If you define $$P=M-I$$ then another of looking at the result is that the vector $c$ is in the nullspace of $P$, i.e. $$\eqalign{ Pc &= 0 \cr c &= (I-P^+P)\,a\cr }$$ where $P^+$ is the pseudoinverse of $P$ and $a$ is an arbitrary vector.

Once you have the vector $c$ you can reshape it (unstack the columns) to recover the matrix $C$.

Update: Nowhere in the above solution was $C$ constrained to be circulant. I assume that means that there are a number of possible vectors which satisfy the eigenvalue equation, and only some of them (none of them?) are associated with a circulant matrix $C$.

Answer 2

$A,B,C$ are assumed to be invertible ciculant matrices. Then $A^T,B^T,C^T$ are also circulant. Morover, all the circulant matrices are diagonalizable in a common complex orthonormal basis; therefore, these matrices pairwise commute and we can define an ordered spectrum of such a matrix.

$C^T=A^{-1}CB^T$ implies $C=BC^TA^{-T}=(BA^{-1})C(B^TA^{-T})=(BA^{-1})(BA^{-1})^TC$. If $C$ is invertible, then $(BA^{-1})(BA^{-1})^T=I$, that is,

(*) $BA^{-1}$ is an orthogonal matrix and also a circulant matrix.

If (*) is not true, then $C$ does not exist or is not invertible.

If (*) is true, then $C^T=B^TA^{-1}C$. Let $(a_j),(b_j),(c_j)$ be the ordered spectra of $A,B,C$.

Note that $|a_j|=|b_j|$. Then $\overline{c_j}=\dfrac{\overline{b_j}}{a_j}c_j$, that defines $c_j$ up to a real factor.