Derivative of trace of product matrix and Frobenius norm

Question

Let $S$ be $n$ x $m$ and $U$ be $m$ x $t$ , We know that

$||SU||^2_F$ =$||S||^2_F ||U||^2_F$

and

$||SU||^2_F$ = Tr$(S^TSUU^T)$

Now, how would I find the derivative of Tr$(S^TSUU^T)$ w.r.t to U?

In other words How can I find $\frac{d}{dU}Tr(S^TSUU^T)$ or $\frac{d}{dU}||SU||^2_F$ equal to?

Answer 1

Use a colon as a convenient product notation for the trace, i.e. $\,A:B = \operatorname{Tr}(A^TB)$
to rewrite the function and calculate its gradient. $$\eqalign{ \phi &= \|SU\|^2_F = SU:SU \\ d\phi &= 2SU:S\,dU = 2S^TSU:dU \\ \frac{\partial\phi}{\partial U} &= 2S^TSU \\ }$$ The cyclic property of the trace allows terms in a colon product to be rearranged in many ways, e.g. $$\eqalign{ A:B &= B^T:A^T \;= B:A \\ A:BC &= B^TA:C \,= AC^T:B \\ }$$ and makes the differential of this product particularly nice $$\eqalign{ d(A:A) &= dA:A + A:dA \\ &= 2A:dA \\ }$$