Derivative of $vec(AA^T)$ w.r.t. $A$, where $A$ is a vector

Question

Let $A$ a $n \times 1 $ vector. What is the derivative of $ vec(AA^T) $ w.r.t. $A$?

Update: I started from $vec(AA^T) = vec(I_n A A^T I_N) = (A \otimes I_N) vec(A) = vec(I_N \otimes A) vec(A^T)$.

Applying the chain rule, the derivative w.r.t $vec(A)$ equals $(A \otimes I_N) + vec(I_N \otimes A) T$, where $T$ is an operator such that $vec(A^T) = T vec(A)$.

However, what I want is the derivative w.r.t. $A$, not $vec(A)$. The problem appears kind of trivial, but I cannot wrap my head around it.

Answer 1

$\newcommand\R{\mathbb R}$If you consider the map \begin{align*} m \colon \R^{n} &\longrightarrow \R^{n\times n}, \\ \mathbf v &\longmapsto \mathbf v\mathbf v^T, \end{align*} its total derivative at $\mathbf v$ is the linear map \begin{align*} \mathrm D_{\mathbf v} m \colon \R^{n} &\longrightarrow \R^{n\times n}, \\ \mathbf z &\longmapsto \mathbf v\mathbf z^T + \mathbf z\mathbf v^T. \end{align*} Vectorizing this result you have the map $\widetilde m\colon \R^{n}\to \R^{n^2}$ sending $\mathbf v$ to $\operatorname{vec}(\mathbf v\mathbf v^T)$ and its total derivative at $\mathbf v$ is given by

$$ \mathbf z \longmapsto \operatorname{vec}(\mathbf v\mathbf z^T + \mathbf z\mathbf v^T). $$

Informally speaking, when $\mathbf x = \operatorname{vec}(\mathbf v\mathbf v^T)$, you have $$ \mathrm d\mathbf x = \operatorname{vec}\Big(\mathbf v (\mathrm d \mathbf v)^T + (\mathrm d \mathbf v) \mathbf v^T\Big). $$ Using Kronecker products you might write $\operatorname{vec}(\mathbf v\mathbf w^T)$ as $\mathbf v\otimes \mathbf w$ so that $\mathbf x = \mathbf v\otimes \mathbf v$ and $$ \mathrm d\mathbf x = \mathbf v\otimes \mathrm d\mathbf v + \mathrm d \mathbf v \otimes \mathbf v. $$

Answer 2

$\def\v{{\rm vec}}\def\d{{\rm diag}}\def\D{{\rm Diag}}\def\p{{\partial}}\def\grad#1#2{\frac{\p #1}{\p #2}}\def\hess#1#2#3{\frac{\p^2 #1}{\p #2\,\p #3^T}}$To minimize confusion, let's use lowercase letters to denote vectors, i.e. $A\to a$.

Expand the vec() operator using Kronecker products in two different ways.
Then calculate its differential and gradient. $$\eqalign{ f &= \v(aa^T) \\ &\doteq \left(I\otimes a\right)a \;=\; \left(a\otimes I\right)a \\ df &= \left(I\otimes a\right)da + \left(a\otimes I\right)da \\ &= \left(I\otimes a+a\otimes I\right)da \\ \grad{f}{a} &= I\otimes a+a\otimes I \\ }$$ Note that, unlike a matrix, for a vector argument $\;\v(a^T)=\v(a)$.