Derivative of d-variable function.

Question

I have this variable function Where is a , and . I wanted to find the derivative at of $f(\textbf{x})$.

---

Answer 1

The place where you went wrong is in the last step of the partial derivative of the quadratic term. You indeed have $$\frac{\partial(\mathbf{x}^T\mathbf{Ax})}{\partial x_i} = \mathbf{e}_i^T\mathbf{Ax} + \mathbf{x}^T\mathbf{Ae}_i$$ but it does not simplify to $2a_{ii}$. In fact, both terms are scalars and therefore are equal to their transpose. As a result, you instead have $$\frac{\partial(\mathbf{x}^T\mathbf{Ax})}{\partial x_i} = \mathbf{e}_i^T\mathbf{Ax} + (\mathbf{x}^T\mathbf{Ae}_i)^T = \mathbf{e}_i^T(\mathbf{Ax} + \mathbf{A}^T\mathbf{x}) = \mathbf{e}_i^T(\mathbf{A} + \mathbf{A}^T)\mathbf{x}$$

For the linear term, you indeed have $\frac{\partial(\mathbf{b}^T\mathbf{x})}{\partial x_i} = b_i = \mathbf{e}_i^T\mathbf{b}$.

Combining both gives you

$$ \frac{\partial f}{\partial x_i}(\mathbf{x}) = \mathbf{e}_i^T((\mathbf{A} + \mathbf{A}^T)\mathbf{x} + \mathbf{b})$$

which in turn yields

$$\nabla_{\mathbf{x}} f (\mathbf{x}) = (\mathbf{A} + \mathbf{A}^T)\mathbf{x} + \mathbf{b}$$

This is true even when $\mathbf{A}$ is not symmetric. It obviously simplifies to $\nabla_{\mathbf{x}} f (\mathbf{x}) = 2\mathbf{A}\mathbf{x} + \mathbf{b}$ when it is symmetric.

P.S.: A good reference for linear algebra using matrices and matrix calculus is the Matrix Cookbook by Petersen and Pedersen (https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf)