Suppose we have a matrix $X$ with $n$ rows and $k$ columns. Further assume that rank of $X$ is $k$ (which means for this particular matrix we must have $n\geq k$). Now the matrix $XX^T$ has $n$ rows and $n$ columns and the rank of $XX^T$ can at most be $k$. So that should mean that $XX^T$ is not invertible (or determinant of $XX^T$ should be zero). Therefore, $XX^T$ can not be a positive semidefinite matrix. But when I read the solution for problem 2.13 of convex optimization book (by Stephen Boyd) I see that they say $XX^T$ is positive definite and its rank is $k$. How is it possible. What is wrong with my thinking here?
Any help in clarifying this confusion will be much appreciated. Thanks in advance.
Positive semi definite matrices can have zero eigenvalues and be non invertible. Positive definite matrices have only positive eigenvalues and are invertible always. A matrix of the form $M^TM$ is going to be positive semi definite regardless of rank of $M$.