For function $$J: \Bbb{R}^d\to\Bbb{R}$$ defined in the following way: $$ J(x) = c + b^Tx + x^TAx $$ where:
- c is a scalar
- b is d-dimensional real vector
- A is a positive definite matrix
How to find gradient and hessian of function J? Any tips or a simple example with d=2 is much appreciated
If $J(x) = c + b^Tx + x^TAx$, then $$DJ(x)(v) = b^Tv + v^TAx+x^TAv = (b+2Ax)^Tv,$$since $(v^TAx)^T=v^TAx$ (it is a real number). This means that $$\nabla J(x) = b+2Ax.$$The Hessian is the total derivative of the above, so $${\rm Hess} J(x)(v,w) = v^T (2A)w = 2v^TAw.$$