Let:
- $x$ be a vector of dimension $n \times 1$.
- A be a matrix of dimension $n \times m$.
I want to find the derivative of $x^T A$ w.r.t. $x$.
By both the numerator and denominator conventions, the derivative of $x^T$ w.r.t. $x$ seems to be the $n \times n$ identity matrix.
By the product rule, $d(x^T \cdot A) = d(x^T) \cdot A + x^T \cdot d(A) $.
$d(A)$ is $0$, and since $d(x^T) = I_n$, the expression should reduce to just $A$.
But Wikipedia claims it's actually $A^T$.
What did I miss/do wrong?
Notice that $x^TAx$ is a function $\mathbb{R}^n\to \mathbb{R}$ so the claim that the derivative is $A^T$ does not make sense. (Also $A$ has to be square for this to make sense.)
Now, $$f(x)=x^TAx=\sum\limits_{i,j}a_{i,j}x_ix_j $$ and so differentiating with respect to $x_i$ gives us $$ \frac{\partial}{\partial x_i}f(x) = \sum_{j=1}^na_{i,j}x_j + \sum_{j=1}^na_{j,i}x_j= e_i^T(A+A^T)x.$$
The derivative is then $\nabla f = x^T(A^T+A).$