Given the matrix A and vector X, what is the partial derivative of AX with respect to A?

Question

If A has the dimension [m$\times$n] and X [n$\times$1], $\frac{\partial AX}{\partial A}$ is said to be $X^T$ (according to Section 2.3.1 here). However, shouldn't $\frac{\partial AX}{\partial A}$ have the dimension [m$\times$m$\times$n] instead of [1$\times$n]? How is this result derived?

Answer 1

In index notation $$\eqalign{ y_i &= A_{in}x_n \cr dy_i &= dA_{in}x_n \cr \frac{\partial y_i}{\partial A_{jk}} &= \delta_{ij}\delta_{nk}x_n = \delta_{ij}x_k \cr\cr }$$ You could write this result using the dyadic $(\star)$ product $$\eqalign{ \frac{\partial y}{\partial A} = I\star x \cr\cr }$$ Or by using the $4^{th}$ order $({\mathbb E}_{ijkl} = \delta_{ij}\delta_{kl})$ isotropic tensor $$\eqalign{ \frac{\partial y}{\partial A} &= {\mathbb E}\,x \cr }$$but neither of these is common.