Simple definition of a positive definite matrix

Question

Aside from what it is written in the books about the definition of a positive definite matrix, can anyone explain it in a more earthly method?

Answer 1

The easiest way to think about it is using eigenvalues. Remember that an eigenvector of the matrix $A$ is a vector $v$ such that

$$ Av = \lambda v $$ for some $\lambda\in\Bbb{R}$ (assuming real vector spaces and not complex). This says that when you act on $v$ with the matrix $A$, $v$ is just scaled by a constant factor. This number can be positive, negative, or zero. If it's positive, the vector is scaled but not reflected. If it's negative, it's scaled and flipped. If it's zero, the vector is in the null space, so it gets sent to zero. The point of a positive definite matrix is that it scales all its eigenvectors positively, and doesn't flip them or send them to zero. This also means that orientations are preserved, i.e. the matrix only stretches and compresses things without flipping them.

Edit: Just for completeness, let's also talk about the standard definition. A matrix $A$ is positive definite if $\langle x,Ax\rangle = x^TAx>0$ for every $x$. This is exactly the "orientation preserving" property: if you think about the inner product/dot product as giving you the cosine of the "angle" between vectors, saying that the dot product of $x$ and $Ax$ is positive means this angle $\theta$ satisfies

$$ -\frac{\pi}{2}<\theta<\frac{\pi}{2} $$ Intuitively, this means that the two vectors are on the same side of the hyperplane $x^\perp$ defined by $x$, see this picture:

A positive definite matrix will have this property for all vectors $x$.