How does the Kronecker product relate to the Jacobian of a matrix?

621 Views Asked by At

I'm working my way through Random Matrix Theory by Edelman and Rao (https://web.eecs.umich.edu/~rajnrao/Acta05rmt.pdf). One of the examples in the paper states that (Ex 1, pg 5):

Let $f(A) = A^2$ so that d$f(E) = EA + AE$. This can be rewritten in terms of the Kronecker product operator $\otimes$ as d$f = I \otimes A + A^T \otimes I$.

where $A$ is some matrix, $E$ is the error, and d$f$ is the Jacobian.

I don't understand how the Jacobian can be rewritten in terms of that Kronecker product equation. Any help would be appreciated.

1

There are 1 best solutions below

0
On

Following the comments: $\DeclareMathOperator\vec{vec}$ $\DeclareMathOperator\Jac{Jac}$

Outline: First compute the differential of matrix multiplication, then use the chain rule to deduce the differential of squaring.

We take the standard basis of $n\times m$ matrices in which the coordinate vector of a matrix $A$ is given by the vectorization $\vec(A)$. For $X\in M^{p\times q}$ and $Y\in M^{q\times r}$ we have $$\vec(XY) = \begin{cases}(Y^T\otimes I_p)\vec(X) \\ (I_r\otimes X)\vec(Y) \end{cases}$$ so that $*: M^{p\times q} \times M^{q\times r} \to M^{p\times r}$ has Jacobian matrix $$(\Jac*)_{(X,Y)}=\begin{pmatrix}Y^T\otimes I_p & I_r\otimes X\end{pmatrix}$$ For $X\mapsto X^2$, use the chain rule: $\Jac(X\mapsto X^2) = (\Jac*)_{(A,A)} \cdot \Jac(X\mapsto(X,X))_A$ (all in the vectorization basis!) to get $$\Jac(X\mapsto X^2)_A = (\Jac*)_{(A,A)}\cdot\begin{pmatrix}I_{n\times n}\\I_{n\times n}\end{pmatrix} = A^T\otimes I_n+I_n\otimes A$$

De-vectorizing we get the familiar $d(X\mapsto X^2)_A(E) = EA+AE$.

(I distinguish between the Jacobian and the differential (as a linear map) to avoid confusion.)

To be complete, the differential of $*$ becomes: $$(d*)_{(X,Y)}\begin{pmatrix}E\\F\end{pmatrix} = EY+XF$$