How do I know the Hessian of $A^TA$ is the minimum for Maximum Likelihood Estimation of a multivariate normal distribution?

159 Views Asked by At

In Maximum Likelihood Estimation for a multivariate normal distribution, I get the following result after the first derivative with respect to $w$: $$\dfrac{\partial \log L}{\partial w}= (y-Aw)^T(y-Aw) = A^Ty + A^TAw$$ If we take the second derivative with respect to $w$, I get the following result: $$\dfrac{\partial^2 }{\partial w\partial w} A^Ty + A^TAw= A^TA$$

I understand how the algebraic steps are done and also I have been told that "$A^TA$ is semi-positive definite since it is invertible. Thus, $A^TA$ is strictly convex."

What I don't really get is that after all the calculations, I just have the above result to see if the result of the second derivative is actually the minimum but how do I know it is the minimum? All I have been told was that $A^TA$ has to be positive semi-definite. It sound like I don't understand the meaning of positive definite and positive semi-definite.

What do we mean by Convex, here? and the meanings of positive definite and positive semi-definite? What do all the words like "convex, minimum, and positive definite" have to do with $A^TA?$ Hope to hear some explanations.