What is the the derivatives of the following problem？

Question

What is the the derivatives of the following problem?

$f(W)$=$tr(e_a^T(diag((WW^T)^{1/2})W)e_b)$ and $f(W)$=$tr(e_a^T(diag((WW^T)^{-1/2})W)e_b)$

where $e_a$ and $e_b$ are two unit column vector. $W\in\mathbb{R}^{n\times m}$.

Thank you for your help.

Answer 1

Define the matrices $(B,E)$ such that $$\eqalign{ &E = e_ae_b^T \cr &B^2 = {\rm Diag}(WW^T) \cr &B\,dB+dB\,B = {\rm Diag}(W\,dW^T+dW\,W^T) \cr &2B\,dB = 2\,{\rm sym}\Big({\rm Diag}(dW\,W^T)\Big) \cr &dB =B^{-1}{\rm sym}\Big({\rm Diag}(dW\,W^T)\Big) \cr }$$ where the funtion $\,{\rm sym}(X)=\tfrac{1}{2}(X+X^T)$

Note that $(B,dB)$ are diagonal and therefore commute with each other.

Write the function in terms of these matrices. Then find its differential and gradient. $$\eqalign{ f &= E:BW \cr df &= E:B\,dW + E:dB\,W \cr &= BE:dW + EW^T:dB \cr &= BE:dW + EW^T:B^{-1}{\rm sym}\Big({\rm Diag}(dW\,W^T)\Big) \cr &= BE:dW + {\rm Diag}\Big({\rm sym}(B^{-1}EW^T)\Big)W:dW \cr &= \Big(BE + \tfrac{1}{2}{\rm Diag}(B^{-1}EW^T + WE^TB^{-1})W\Big):dW \cr \frac{\partial f}{\partial W} &= BE + \frac{1}{2}{\rm Diag}\big(B^{-1}EW^T + WE^TB^{-1}\big)W \cr }$$ In the above, the trace/Frobenius product is represented by a colon, i.e. $$\eqalign{A:B = {\rm Tr}(A^TB)}$$

Update
The Diag() operation in the final result can be simplified to yield $$\frac{\partial f}{\partial W} = BE + B^{-1}{\rm Diag}\big(EW^T\big)W$$