Matrix gradient of linear form

Question

My question is very basic, and I ask it because I haven't seem to have found it online. What is the matrix gradient (with respect to A) of Ax, where A is m by n and x is n by 1?

Answer 1

Let ${\bf E}$ represent the 4th order tensor $$\eqalign{ {\bf E} &= \frac {\partial A} {\partial A} \cr {\bf E}_{ijkl} &= \delta_{ik} \delta_{jl} \cr }$$ Then the derivative of $f=A x$ is a 3rd order tensor, which can be written as $$\eqalign{ \frac {\partial f} {\partial A} &= {\bf E}\,x \cr \frac {\partial f_i} {\partial A_{kl}} &= \delta_{ik} \delta_{jl} x_j \cr &= \delta_{ik} \delta_{lj} x_j \cr }$$