Why the result of matrix derivative is its transpose?

Question

I'm having trouble understanding my lecture notes.

Given a matrix $A\in\mathbb R^{n\times n}$ and vectors $u,f\in\mathbb R^n,$ if we calculate the derivative (w.r.t. u) to of $Au-f$ the result will be $A^T.$

Why is it $A^T$ and not just $A$?

Answer 1

For a map $g : \mathbb{R}^n \to \mathbb{R}^n$, it just depends on how you define the Jacobian matrix: is the $(i,j)$ entry $\frac{\partial g_i}{\partial u_j}$ or $\frac{\partial g_j}{\partial u_i}$?

• Wikipedia seems to prefer the former, in which case if $g(u) = Au-f$, then $g_i(u) = \sum_{j=1}^n a_{ij} u_j - f_i$ so the $(i,j)$ entry of the Jacobian is $\frac{\partial g_i}{\partial u_j} = a_{ij}$, i.e. the Jacobian is $A$.
• But Wikipedia also notes that some authors prefer the latter, where the $(i,j)$ entry of the Jacobian is $\frac{\partial g_j}{\partial u_i} = a_{ji}$, i.e. the Jacobian is $A^\top$.