I have the following formula
$F(X) = \text{vec}(Z^T(A-B-ZXX^TZ^T)ZX)$
and I want to take the derivative of it with respect to the matrix $X$. What is the proper way to do this? I believe the first two terms are
$(I \otimes Z^TAZ) - (I \otimes Z^TBZ)$ but having trouble with the third term. Any help would be greatly appreciated! Thanks
Where vec is the vec operator, that takes a matrix and puts its contents into a vector column-wise.
$Z$ is $n \times m$
$A$ is $n \times n$
$B$ is $n \times n$
$X$ is $m \times m$
For convenience, define some new variables. $$\eqalign{ C &= A-B \\ Y &= Z^TCZ-Z^TZXX^TZ^TZ \\ f &= \operatorname{vec}(F),\quad x = \operatorname{vec}(X) \\ }$$ Write the function in terms of these new variable, calculate its differential, and vectorize it. $$\eqalign{ F &= Z^TCZX-Z^TZXX^TZ^TZX \\ dF &= Z^TCZ\,\color{red}{dX} -Z^TZ\,\color{red}{dX}\,X^TZ^TZX -Z^TZX\,\color{red}{dX^T}Z^TZX -Z^TZXX^TZ^TZ\,\color{red}{dX} \\ &= Y\,\color{red}{dX} -Z^TZ\,\color{red}{dX}\,X^TZ^TZX -Z^TZX\,\color{red}{dX^T}Z^TZX \\ df &= (I\otimes Y)\,dx -(X^TZ^TZX\otimes Z^TZ)\,dx -(X^TZ^TZ\otimes Z^TZX)K\,dx \\ &= \Big(I\otimes Y - X^TZ^TZX\otimes Z^TZ - (X^TZ^TZ\otimes Z^TZX)K\Big)\,dx \\ }$$ where $K$ is the Commutator matrix associated with the Kronecker product.
Finally, calculate the gradient matrix and revert to the original variables. $$\eqalign{ \frac{\partial f}{\partial x} &= \Big(I\otimes Y - X^TZ^TZX\otimes Z^TZ - (X^TZ^TZ\otimes Z^TZX)K\Big) \\ &= I\otimes(Z^TAZ-Z^TBZ-Z^TZXX^TZ^TZ) - X^TZ^TZX\otimes Z^TZ - (X^TZ^TZ\otimes Z^TZX)K \\ }$$