Descent direction with symmetrically positive definite hessian matrix and $∇f(x_{k})=0$ with $Hf(xk)d_{k}=−∇f(x_{k})$

Question

I have the following problem which I can't figure out how to solve it.

Let $f:R^n→R$ be twice continuously differentiable. Furthermore, let $d_{k}$ denote the solution of the Newton equation $Hf(xk)d_{k}=−∇f(x_{k})$ for the Hessian matrix $Hf(x_{k})$ and the gradient $∇f(x_{k})$. I must show that $d_{k}$ defines a descent direction at $f$ at $x_{k}$ if $Hf(x_{k})$ is symmetrically positive definite and $∇f(x_{k})=0$.

Can someone help me?

Answer 1

We say that $d_k$ is a descent direction if $\langle d_k,\nabla f(x_k)\rangle\leq 0$, this is because a Taylor expansion yields, $$ f(x_k+d_k) = f(x_k) + \langle d_k,\nabla f(x_k)\rangle + o(\Vert d_k\Vert),$$ thus up to a first-order approximation (which must be treated carefully), $f(x_k+d_k) \leq f(x_k)$, if $\langle d_k,\nabla f(x_k)\rangle\leq 0$.

In your case, $ \langle d_k,\nabla f(x_k)\rangle = - \langle H_f(x_k)\nabla f(x_k),\nabla f(x_k)\rangle$. This quantity is (strictly) negative if $H_f(x_k)$ is (definite) positive, by definition of a positive matrix. If you want to convince yourself of that, you can diagonalize $H_f(x_k)$ in an orthogonal basis.