Why is the hessian of strictly convex quadratic functions positive definite?

288 Views Asked by At

Quadratic functions are of the form: $$f(x)=b^Tx+\frac{1}{2}x^TCx$$ where $C$ is assumed without loss of generality to be symmetric. Tried to prove it myself using contradiction but I didn't find a valid argument.

2

There are 2 best solutions below

0
On BEST ANSWER

If $f$ is strictly convex then for $x\neq y$ we have $f(y)> f(x) + \nabla f(x)(y-x)$. But by 2nd order Taylor we have that $f(y)=f(x) + \nabla f(x)(y-x) + (y-x)^T \nabla^2 f(x+\alpha(y-x)) (y-x)$ for some $0 \leq \alpha \leq 1$. Now given that $f$ is quadratic then $\nabla^2 f(z) = C$ for all $z$, hence we have that: $$(y-x)^T C (y-x) > 0$$ for all $y\neq x$, since these are arbitrary it follows that $C$ is positive definite.

0
On

If $C$ is not positive definite, there is an eigenvalue-eigenvector pair $(\lambda,v)$ where $\lambda\leq 0$.

Consider now the function $f(x)$ evaluated along the direction of $v$ (where $t\in\mathbb{R}$):

$f(tv) = (b^Tv)t + (\lambda v^Tv)t^2$

This is the equation of a line or of a parabola upside down, hence not strictly convex