Proving least squares is a minima

Question

Minimizing $ f(\beta) = (Y-X\beta)'(Y-X\beta) $ we take the derivative set it to zero and get $ \beta = (X'X)^{-1}X'Y $ now everywhere I've read they say this is the minima because $ (X'X) $ is positive definite, if its full rank. Now why does that tell us this is a minima? Don't we need to retrieve the hessian matrix or something?

Answer 1

The Hessian matrix is $2 X' X$. But this is taking things slightly backwards. The idea of the Hessian matrix is to approximate your function in the neighbourhood of the point in question by a quadratic form. Here you already have a quadratic form to begin with.

One of the characterizations of positive definite is: an $n \times n$ real symmetric matrix $A$ is positive definite if and only if $v^T A v > 0$ for all $v \in \mathbb R^n$ except $0$.