Kronecker Product Expression Related to Matrix Differential

Question

The equivalence of the below expressions is well documented (see, e.g., Brewer (1978). Kronecker Products and Matrix Calculus In System Theory, IEEE Trans on Circuits and Systems, CAS-25(9) 772:781, Sep 1978): for $\underline{A} \in \mathbb{R}^{p \times q}$ and $\underline{D} \in \mathbb{R}^{q \times s}$, $$ (\underline{I}_s \otimes \underline{A})\, \text{vec}[\underline{D}] = (\underline{D}^T \otimes \underline{I}_p)\, \text{vec}[\underline{A}] = (\underline{D}^T \otimes \underline{A})\, \text{vec}[\underline{I}_q] = \text{vec}[\underline{A}\, \underline{D}]. $$ These require that $\underline{A}$ and $\underline{D}$ be compatible matrices for multiplication. What if they are not compatible? Are there equivalences that are analogous to the above in such a case?

I am interested in this because of the following equation that I have encountered: $$ (\underline{I}_N \otimes \underline{x}^T)\, \frac{\partial \underline{B}^T}{\partial \underline{x}}, $$ where $\underline{x} \in \mathbb{R}^{N \times 1}$ and $\underline{B} \in \mathbb{R}^{N \times N}$, meaning that $(\underline{I}_N \otimes \underline{x}^T) \in \mathbb{R}^{N \times N^2}$ and $\partial \underline{B}^T/\partial \underline{x} \in \mathbb{R}^{N^2 \times N}$. Note that, $\underline{x}^T$ and $\partial \underline{B}^T/\partial \underline{x}$ are not compatible.

Any advice/suggestion/comment is most appreciated. Thanks.

Answer 1

Non-compatible equalities are not allowed. Therefore, equalities 1 and 2 hold and 3 do not. You must add a compatibility matrix by substution of identity matrix by some rectangular matrix. Take a look at Theorem 1 here.

$$\mbox{vec}(AXB) = (B^\intercal \otimes A) \mbox{vec}(X)$$

I will call $\frac{\partial B}{\partial x}$ as matrix $B_x$ and each column of $B_x^\intercal$ as column vector $c_i$. Therefore, we use the above identity for expression $(I_n \otimes x^\intercal) \, c_i$. The vector $c_i$ has dimension $n^2 \times 1$, thus vec$(c_i)$ emerges an square matrix $C_i$. Finally, the desired result $(I_n \otimes x^\intercal) \, B_x$ corresponds to the column composition of column vectors $\mbox{vec}(x^\intercal C_i)$, for index $i$ from 1 to $n$. $\blacksquare$.

Answer 2

Thank you to all who responded to this question. I have now been able to establish the following identities:

Given $\underline{x} \in \mathbb{R}^{N \times 1}$ and $\underline{A} \in \mathbb{R}^{N \times N}$, $$ \frac{\partial \underline{A}}{\partial \underline{x}^T} \centerdot [\underline{I}_N \otimes \underline{x}] = \left( \underline{x}^T \otimes \underline{I}_N \right) \frac{\partial [\text{vec}[\underline{A}]]}{\partial \underline{x}^T} = [\underline{I}_N \otimes \underline{x}^T] \centerdot \frac{\partial [\text{vec}[\underline{A}^T]]}{\partial \underline{x}^T}, $$ where $\text{vec}[\cdot]$ denotes stacking the columns on top of each other. Note that, if you transpose these identities, we get identities for $$ [\underline{I}_N \otimes \underline{x}^T] \centerdot \frac{\partial \underline{A}^T}{\partial \underline{x}}, $$ which is the expression I mentioned in my initial post.

If you have come across these identities, please let me know the location where they appear; or, if you feel there is something wrong in these identities, again, please let me know. Thanks.