Matrix derivative of $x^\top A$ with respect to $A$

Question

I'm interested in the following notation-problem. Assume we have a transpose vector equals $x^\top A$, where $x^\top \in \mathbb{R}^n$ and $A \in \mathbb{R}^{n \times m}$. I'm interested in $\dfrac{d (x^\top A)}{d A}$. I understand that it will be 3rd order tensor. But does there any chance (or operation) to represent it with a close form?

Answer 1

First, I don't like multiplying by vectors on the left, so I will talk about $y=A x$ (transpose at your own peril). Second, we just need to compute:

$$\frac{\partial y_i}{\partial{a_{kl}}}$$ Notice that $y_i = \sum x_j a_{ij},$ so the partial in question is equal to $x_l$ if $i=k,$ and $0$ otherwise.

Answer 2

The components of $x^T A$ are $(x^T A)_j = x_i A_{ij}$ using Einstein summation convention on the index $i$. Using the fact that $ \frac{\partial A_{ij}}{\partial A_{lm}} = \delta_{i,l} \delta_{j,m}$, you can see that \begin{equation} \frac{\partial (x_i A_{ij})}{\partial A_{lm}} = x_i \delta_{i,l} \delta_{j,m}= x_l \delta_{j,m} \end{equation}which, as you predicted, is a tensor with three indices.