Hessian Matrix: Meaning is definite

Question

Why does the Hessian matrix being positive definite mean it is a minimum point, and negative definite mean it is a maximum point?

Answer 1

Taylor's theorem for a function $f\in C^2({\mathbb R}^n,{\mathbb R})$ says that $$f(x)=f(0)+\nabla f(0)\cdot x+ x^\top Hx +o(|x|^2)\ ,$$ where $H$ is the Hessian of $f$ at $0\in{\mathbb R}^n$. In the context of this problem it is assumed that $\nabla f(0)=0$. If $H$ is positive definite then $x^\top Hx\geq\alpha |x|^2$ for some $\alpha>0$, and this implies $$f(x)-f(0)=x^\top H x+o(|x|^2)\geq \alpha|x|^2+o(|x|)^2>0\qquad\bigl(0<|x|<\epsilon)\ .$$ This shows that $f(x)-f(0)>0$ for $0<|x|<\epsilon$, hence $f$ assumes a local minimum at $0$. – Similarly when $H$ is negative definite.