trace and derivative: understanding $\text{tr}\left(e_j e_i^T B^T B \right) = \langle B e_i e_j^T, B \rangle $

Question

where $B$ is $n \times n$ matrix

Can someone help me understand the above equality. what kind of inner product we are using here?

Also, why do we have $\frac{d}{dB}\left(\langle B e_i e_j^T, B \rangle \right) = B e_i e_j^T $ ?

Answer 1

The Frobenius product (aka Inner product) can be written in terms of the trace $$A:B={\rm Tr}\big(AB^T\big)$$ The trace's cyclic property means that such products can be rearranged in various ways, e.g. $$\eqalign{ A:B &= A^T:B^T &= B:A \\ A:BC &= AC^T:B &= B^TA:C\\ }$$ Write the function in terms of the Frobenius product. Then calculate its differential and gradient. $$\eqalign{ \phi &= {\rm Tr}\left(e_je_i^TB^TB\right) \\ &= e_je_i^T:B^TB \\ d\phi &= e_je_i^T:(dB^TB+B^TdB) \\ &= (e_je_i^T)B^T:dB^T + B(e_je_i^T):dB \\ &= B(e_je_i^T)^T:dB + B(e_je_i^T):dB \\ &= B(e_ie_j^T+e_je_i^T):dB \\ \frac{\partial\phi}{\partial B} &= B\left(e_ie_j^T+e_je_i^T\right) \\ }$$