Finding a solution to matrix equation occurring inside an optimization problem

Question

As a part of an optimization problem (while equating the derivative of the cost function to 0), I'm getting the following expression. $$-2XX^TC + 2XX^TACC^T + \gamma GA = 0,$$ where, $X, C, G$ are constant matrices with G being a diagonal matrix. $\gamma$ is a constant scalar. Can the unknown matrix $A$ be expressed in terms of these constants, especially keeping in mind that $XX^T$ and $CC^T$ are symmetric positive semi-definite? For the purpose, $XX^T$, $CC^T$ and $G$ can be taken as invertible

Answer 1

The answer to your question is yes! You can do this with the help of the Kroecker product, see http://en.wikipedia.org/wiki/Vectorization_(mathematics).

Your equality has the form $$ P A R + S A = Q, $$ where $Q=2XX^TC $, $P=2XX^T $, $R = CC^T$, and $S = \gamma G $. Then, The vectorization ${\rm vec}$ can be used together with the Kronecker product to express the matrix multiplication as a linear transformation on matrices: $$ {\rm vec}\{ P A R + S A\} = {\rm vec}\{ Q \}. $$ We use the following identities $${\rm vec} \{ P A R \} = (R^T \otimes P) {\rm vec}\{ A \}, $$ $${\rm vec} \{ SA \} = (I \otimes S) {\rm vec}\{ A \}, $$ where $\otimes$ denotes the Kronecker and $I$ is the identity matrix of appropriate size. Finally, we have

$$ {\rm vec}\{ A \} = ((R^T \otimes P) + (I \otimes S) )^{-1} {\rm vec}\{ Q \}. $$

Using the above equality, we derive the vector ${\rm vec}\{ A \}$ which gives us (after stacking the vector entries together to matrix) $A$.