Let's define function $f : \mathbb{R}^n \to \mathbb{R}$ as
$$ f(x) = {1\over2}x'Ax + b'x $$
where matrix $A \in \mathbb{R}^{n\times n}$ and vector $b\in \mathbb{R}^n$ are given. Function $f$ is twice differentiable.
How can one prove the following?
$$\nabla f(x) = {1\over 2} (A+A')x + b$$
Well you could expand it as $$f(x) = \frac{1}{2} \sum_i \sum_j x_ix_j A_{ij} + \sum_i b_ix_i$$ Derive with respect to $x_k$
$$\frac{\partial}{\partial x_k} f(x) =\frac{1}{2}(\sum_j x_j A_{kj} + \sum_i x_i A_{ik} ) + b_k$$
This means that
$$ \begin{bmatrix} \frac{\partial}{\partial x_1} f(x)\\ \frac{\partial}{\partial x_2} f(x)\\ \vdots\\ \frac{\partial}{\partial x_n} f(x) \end{bmatrix} = \begin{bmatrix} \frac{1}{2}(\sum_j x_j A_{1j} + \sum_i x_i A_{i1} ) + b_1\\ \frac{1}{2}(\sum_j x_j A_{2j} + \sum_i x_i A_{i2} ) + b_2\\ \vdots\\ \frac{1}{2}(\sum_j x_j A_{nj} + \sum_i x_i A_{in} ) + b_n \end{bmatrix}$$ The first term above it $\frac{1}{2} Ax$, the second is $\frac{1}{2} A^T x$ and the third is $b$.