Derivative of trace of $A^\top A$

Question

Presume you have $$f(A) = \frac{1}{2} \mbox{trace}(A^\top A)$$ with $A$ "nice" enough. What is its derivative $\frac{Df(A)}{DA}$ and how do you compute it?

Answer 1

You have :

If you want to talk about partial derivative here it is more on $ \partial_i \ \forall i \in [| 1, n^2 |] $ I mean you can talk of a function of a variable being a matrix or being $n^2$ coefficients.

So if you want to differentiate your application it gives us :

$$ f(A+H)=\dfrac{1}{2}Tr((A+H)(A+H)^{\top})=\dfrac{1}{2}Tr(AA^{\top}+HA+AH+HH^{\top})$$ Thus $$ f(A+H)-f(H)=\dfrac{1}{2}Tr{(AH^{\top}+HA^\top)}+\dfrac{1}{2}||H||^2=\dfrac{1}{2}Tr(AH^\top+HA^\top) +o(||H||)$$

By defintion of differential in A:

$$ \phi(H)_A = \left< A, H \right> $$

Answer 2

Let

$$f (\mathrm X) := \frac 12 \mbox{tr} \left( \mathrm X^\top \mathrm X \right) =: \frac 12 \| \mathrm X \|_{\text{F}}^2$$

Hence,

$$f (\mathrm X + h \mathrm V) = f (\mathrm X) + \frac h2 \mbox{tr} \left( \mathrm V^\top \mathrm X \right) + \frac h2 \mbox{tr} \left( \mathrm X^\top \mathrm V \right) + O(h^2)$$

and, thus, the directional derivative of $f$ in the direction of $\rm V$ at $\rm X$ is

$$\lim_{h \to 0} \frac{f (\mathrm X + h \mathrm V) - f (\mathrm X)}{h} = \frac 12 \mbox{tr} \left( \mathrm V^\top \mathrm X \right) + \frac 12 \mbox{tr} \left( \mathrm X^\top \mathrm V \right) = \mbox{tr} \left( \mathrm V^\top \mathrm X \right) = \langle \mathrm V, \mathrm X \rangle$$

where $\langle \cdot \,, \cdot \rangle$ is the Frobenius inner product. Thus, the gradient of $f$ is $\nabla f (\mathrm X) = \mathrm X$.

---

matrix-calculus scalar-fields gradient

Answer 3

You get a matrix whose $ij$ entry is $\partial_{A_{ij}}\frac{1}{2}\sum_{kl}A_{kl}^2$. If all components of $A$ are independent, this entry is $A_{ij}$, so the derivative is $A$.