Calculus derivation of OLS regression formula

Question

Edited to make the question more clear.

In deriving the OLS formula from calculus, that is, solving $$\min_{\beta}(Y-X\beta)^T(Y-X\beta).$$ How could be guarantee the solution is actually the minimum? I read something like "because $X$ is full-rank, so $X^TX$ is positive definite". Could anyone show me how to conclude that $X^TX$ is positive definite out of $X$ is full-rank?

Answer 1

$X$ is $n \times k$ where $k\le n$. So $X$ full rank means the rank of $X$ is $k$. Now if $x$ is $k \times 1$, then $y=Xx$ is definite: $Xx=0$ iff $x=0$ by the full rank of $X$ (if not, some $k \times 1$ basis vector $e$ would have $Xe=0$, contradicting full rank). Finally, $x^TX^TXx=y^Ty$ is clearly positive, and so $X^TX$ is positive-definite.

Answer 2

Check the definition of positive definite matrix. You will find that the objective function is convex. And the second order derivative with respect to $\beta$ is $X^TX$. When it is positive definite, like $y=f(x)$, if at $x^{\ast}$ the second derivative is positive, $f(x)$ achieve its minimum.