I have the following question:
For the function
$f=\|XY^\top-A(I\otimes X)Z^\top\|_{F}^{2}$
where $\otimes$ denotes the Kronecker product, and $\|\cdot\|_{F}$ denotes the Frobenius norm.
What is the result of $\frac{\partial f}{\partial X}$?
Thank you in advance for your help!
$\def\R#1{{\mathbb R}^{#1}}\def\v{{\rm vec}}\def\d{{\rm diag}}\def\D{{\rm Diag}}\def\o{{\tt1}}\def\p{{\partial}}\def\grad#1#2{\frac{\p #1}{\p #2}}\def\z{{\rm size}}$For typing convenience, define the following matrix variables $$\eqalign{ R &= XY^T-A(I_p\otimes X)Z^T \\ B &= A^TRZ \\ }$$ and use $e_k$ to denote the $k^{th}$ column of the identity matrix $I_p\,$ in order to define the block matrices $$\eqalign{ &M_k = e_k\otimes I_m \\ &N_k = e_k\otimes I_n \\ }$$ and the matrix sum $$\eqalign{ &S = \sum_{k=1}^p\;M_k^TBN_k\\ }$$
Let's also assign concrete dimensions to all of the matrices involved $$\eqalign{ m,n &= \z(X) = \z(S) \\ q,n &= \z(Y) \\ m,mp &= \z(A) \\ p,p &= \z(I_p),\quad\z(e_k) = p,1 \\ q,np &= \z(Z) \\ m,q &= \z(R) \\ mp,np &= \z(B) \\ mp,m &= \z(M_k) \\ np,n &= \z(N_k) \\ }$$ One final bit of notation is to use a colon to denote the trace/Frobenius product $$\eqalign{ S:X &= \sum_{i=1}^m \sum_{j=1}^n\;S_{ij} X_{ij} \;=\; {\rm Tr}(S^TX) \\ X:X &= \big\|X\big\|^2_F \\ }$$ Now write the objective using the above notation. Then calculate its differential and gradient. $$\eqalign{ f &= R:R \\ df &= 2R:dR \\ &= 2R:\big(dX\,Y^T - A(I_p\otimes dX)Z^T\big) \\ &= 2RY:dX - 2A^TRZ:(I_p\otimes dX) \\ &= 2RY:dX - 2B:(I_p\otimes dX) \\ &= 2(RY-S):dX \\ \grad{f}{X} &= 2(RY-S) \\\\ }$$
The following identity was utilized above $$\eqalign{ S:dX &= \sum_{k=1}^p\; \left(e_k^T\otimes I_m\right)B\left(e_k\otimes I_n\right):dX \\ &= \sum_{k=1}^p\; B:\left(e_k\otimes I_m\right)(I_\o\otimes dX)\left(e_k^T\otimes I_n\right) \\ &= \sum_{k=1}^p\;B:\big(e_ke_k^T\big)\otimes\big(dX\big) \\ &= B:I_p\otimes dX \\ }$$