So let $A \in M_{n}$ and define $f: \mathbb{R}^n \to \mathbb{R}$ by $f(x) = \langle Ax, x \rangle $. Find f' and f''.
After some work, I found the first derivative to be $f'(x)(v) = \langle Ax, v \rangle + \langle Av, x \rangle = \langle (A^T + A)x, v \rangle$.
That is not what I'm having trouble with.
I don't think it is necessary to compute the second derivative the "long way" through a complicated limit.
I believe that since $f'$ is itself a linear function, then we should have $f''(x) = f'$. That $f''(x)(v,w) = f'(v,w) = \langle (A^T + A) v, w \rangle$.
However I don't know how to justify this. I feel that there are some basic mechanics that I am missing. I am more interested in the mechanics than in a direct answer.