How to interpret a second derivative with respect to a vector/matrix?

Question

I found not many resources are devoted to matrix derivatives, esp. when you go beyond order one. Recently I came across this (seemingly very simple) equation in some literature (it would be appreciated if someone can help me make better sense of it):
It is well known that $\frac{\partial{x^T Ax}}{\partial x} = (A^T + A)x$. Now if I do another derivative w.r.t. $x$, I got $\frac{\partial^2{x^T Ax}}{\partial x\partial x^T} = 2A$. My confusion is that how you get $2A$ instead of $A^T + A$ for the second derivative, and in general, when we take second derivative w.r.t. some vectors/matrices, why do we transpose them?