The derivative of a matrix transpose with respect to the original matrix

Question

I am just stepping into matrix calculus and I wonder what the following differential is. Thanks.

$$ \frac{\partial(A^T)}{\partial A} $$

Answer 1

The transposition of a $3 \times 2$ matrix is

$$\begin{bmatrix} x_{11} & x_{12}\\ x_{21} & x_{22}\\ x_{31} & x_{32}\\ \end{bmatrix} \mapsto \begin{bmatrix} x_{11} & x_{21} & x_{31}\\ x_{12} & x_{22} & x_{32}\end{bmatrix}$$

Hence,

$$\dfrac{\partial x_{11}}{\partial \mathrm X} = \begin{bmatrix} 1 & 0\\ 0 & 0\\ 0 & 0\\ \end{bmatrix} \qquad \qquad \dfrac{\partial x_{21}}{\partial \mathrm X} = \begin{bmatrix} 0 & 0\\ 1 & 0\\ 0 & 0\\ \end{bmatrix} \qquad \qquad \dfrac{\partial x_{31}}{\partial \mathrm X} = \begin{bmatrix} 0 & 0\\ 0 & 0\\ 1 & 0\\ \end{bmatrix}$$

$$\dfrac{\partial x_{12}}{\partial \mathrm X} = \begin{bmatrix} 0 & 1\\ 0 & 0\\ 0 & 0\\ \end{bmatrix} \qquad \qquad \dfrac{\partial x_{22}}{\partial \mathrm X} = \begin{bmatrix} 0 & 0\\ 0 & 1\\ 0 & 0\\ \end{bmatrix} \qquad \qquad \dfrac{\partial x_{32}}{\partial \mathrm X} = \begin{bmatrix} 0 & 0\\ 0 & 0\\ 0 & 1\\ \end{bmatrix}$$

Answer 2

The gradient $\,\,{\mathcal H}=\frac{\partial A^T}{\partial A}\,\,$ is a fourth order tensor whose components are given by $$\eqalign{ {\mathcal H}_{ijkl} &= \frac{\partial A^T_{ij}}{\partial A_{kl}} = \frac{\partial A_{ji}}{\partial A_{kl}} \,\,\,= \delta_{jk}\,\delta_{il} \cr }$$ where the Kronecker delta has components $$\delta_{jk}= \begin{cases}1 &\text{if}\,\,\,\,j=k \\ 0 &\text{otherwise} \end{cases}$$