The derivative of Frobenius norm with respect to the transpose of the matrix

60 Views Asked by At

The F-norm is $\|X-T^{\top}Z\|_F^2$,my question is what's the derivative with respect to $T$?

My solution is $$ \begin{aligned} \|X-T^{\top}Z\|_F^2 &= {\rm tr}\Big(\big(X-T^{\top}Z\big)^{\top}\big(X-T^{\top}Z\big)\Big)\\ &={\rm tr}\Big(X^{\top}X-X^{\top}T^{\top}Z-Z^{T}TX+Z^{\top}TT^{\top}Z\Big) \end{aligned} $$ So differentiate with respect to $T$ is $$ 2ZZ^{\top}T-2ZX^{\top}=2Z\big(Z^{\top}T-X^{\top}\big)=2Z\big(T^{\top}Z-X\big)^{\top} $$ But the answer is $$ 2\big(T^{\top}Z-X\big)Z $$ Is the answer right?If so,can you give me a solution,thanks in advance!

1

There are 1 best solutions below

0
On BEST ANSWER

The Frobenius norm is unaffected by negation or transposition, i.e. $$\eqalign{ \def\p{\,\big\|A\big\|_F^2} \def\G{\frac{\partial \p}{\partial T}} \p &= {\rm Tr}(A^TA) &\equiv A:A \\ \big\|-\!A\big\|_F^2 &= (-\!A):(-\!A) &= A:A \\ \big\|A^T\big\|_F^2 &= {\rm Tr}(AA^T) &= A:A \qquad\qquad\qquad \\ }$$ Substituting $\,A=(Z^TT-X^T)\:$ yields the desired gradient $$\eqalign{ dA &= Z^TdT \\ d\p &= 2A:dA = 2A:Z^TdT = 2ZA:dT \\ \G &= 2ZA = 2Z(Z^TT-X^T) \\ }$$ So your calculation is correct.

However, some layout conventions insist that this result should be transposed $$\eqalign{ \G &= 2\,(T^TZ-X)Z^T \qquad\qquad\qquad\qquad \\ }$$