Prove that $\nabla_A \mbox{Tr} \left( A A^T \right) = 2A$

Question

Prove that $$\nabla_A \mbox{Tr} \left( A A^T \right) = 2A$$ where $A$ is any square matrix.

---

I did a simple derivative with product rule, but I don't know where i messed up. I started with

$$ \nabla_A \mbox{Tr} \left( A A^T \right) = \mbox{Tr} \left( \frac {\partial A}{\partial A}A^T+A\frac {\partial A^T}{\partial A} \right) $$

Answer 1

I'm going to work with square matrices, but it should be easy enough to change the dimensions as you please. Write $A = [\vec x_1 | \cdots | \vec x_n ]$ where each $\vec x_i = [x_{1i}, \ldots, x_{ni}]$. Notice then that the $i^{th}$ diagonal element of $AA^T$ is just $\vec x_i \cdot \vec x_i = x_{1i}^2 + \cdots + x_{ni}^2$. Thus the trace of $AA^T$ is the sum of every element of $A$; namely $$ \operatorname{Tr}(AA^T) = \sum_{i=1}^n \sum_{j=1}^n x_{ij}^2.$$ Now taking $\nabla_A$ will give the desired result.

Answer 2

$ \nabla_A(Tr(AA^t))_{kl} = \nabla_A(A_{ij}(A^t)_{ji})_{kl} = \nabla_A(A_{ij} A_{ij})_{kl} = \nabla_A(A_{ij})_{kl}A_{ij}+A_{ij}\nabla_A(A_{ij})_{kl} = 2A_{ij}\delta_{ki}\delta_{lj} = 2A_{kl}$