Computing gradient and hessian of a function

84 Views Asked by At

Let $f$ be a map from $R^n$ to $R$ so that $f(x) = \frac{1}{2} x \cdot (Ax) - x^T v + \alpha $ where $A$ is an $n by n$ symmetric and positive definite matrix. $v \in \mathbb{R}^n$ and $\alpha $ is a scalar. IS there an easier way to compute the gradient and hessian of $f$?

It seems not too complicated but we still have a ltot of equations involved. Any ideas?

1

There are 1 best solutions below

2
On BEST ANSWER

The wikipedia page of matrix calculus is helpful for such stuff.

Note that $A$ is symmetric positive definite.

$$f(x)=\frac12 x^TAx-x^Tv+\alpha$$

$$\nabla f(x)=\frac12 (2Ax)-v=Ax-v$$

If we differentiate by $x$ again, $v$ as a constant vector will vanish and the first part is just linear combination of $x_i$.

$$\nabla^2 f(x)=A.$$

Compare the result with lower dimensions to make sure they agree.