Derivative of the following matrix with respect to a vector

Question

$$J=X^T(Q\otimes I) X+U^T(Z\otimes I)U$$

$\frac{\partial J}{\partial U}=? $

where $\otimes$ is the Kronecker product.

$X$ is a vector which depends on vector $U$.

$Q, Z$ is a positive semi-definite matrix,

$I$ is identity matrix.

superscript $T$ is the transpose of the matrices.

My try:

$\frac{\partial U^T(Z\otimes I)U}{\partial U}=(Z\otimes I)U+(Z\otimes I)^TU $

it is right?

$\frac{\partial X^T(Q\otimes I)X}{\partial U}=?$

Answer 1

You've correctly calculated $$\frac{\partial(U^T(Z\otimes I)U)}{\partial U}=(Z\otimes I)U+(Z\otimes I)^TU\,\,=A$$ Simply replacing $(U,Z)$ by $(X,Q)$ yields
$$\frac{\partial(X^T(Q\otimes I)X)}{\partial X}=(Q\otimes I)X+(Q\otimes I)^TX\,\,=B$$ And although you haven't told us, I assume that you know the relationship between $X$ and $U$ and are able to calculate the matrix $C=\frac{\partial X}{\partial U}$

All that's left is to combine these 3 quantities to obtain the desired gradient. $$\eqalign{ dJ &= A:dU + B:dX \cr &= A:dU + B:(C\,dU) \cr &= (A + C^TB):dU \cr \frac{\partial J}{\partial U} &= A + C^TB \cr }$$ For ease of typing, I used a colon to denote the inner/scalar product.

Answer 2

Derivatives should be understood under the context of linear approximation. Then

\begin{align*} dJ &= (dX)^T (Q \otimes I) X + X^T (Q \otimes I) dX \\ &\quad + (dU)^T ( Z \otimes I ) U + U^T ( Z \otimes I) dU \end{align*}

As for $dX$, we may invoke Jacobian $\operatorname{Jac}(X) = \left( \frac{\partial X^i}{\partial U^j} \right)$ to write $dX = \operatorname{Jac}(X) dU$. Then

$$ dJ = (dU)^T \left[ \operatorname{Jac}(X)^T (Q \otimes I) X + (Z \otimes I) U \right] + \left[ X^T (Q \otimes I) \operatorname{Jac}(X) + U^T (Z \otimes I)\right] dU. $$

If needed, one may read out the partial derivatives $\frac{\partial J}{\partial U^j}$ from the identity above.