Let's suppose I have a table of size m x n. Is a correlation matrix SPD? Why / Why not? (By correlation matrix I mean a matrix M of size mxm where M(i,j) = Pearson correlation between i-th column and j-th column, see https://en.wikipedia.org/wiki/Pearson_correlation_coefficient)
It's easy to prove that M is symmetric, as Pearson(i,j) = Pearson(j,i). However, how do I prove that it is / is not also positive definite?
A covariance matrix $\rho$ of variables $X_i$ is positive-semidefinite because$$v_i\rho_{ij}v_j=\operatorname{Cov}(v_iX_i,\,v_jX_j)=\operatorname{Var}(v_iX_i)\ge0.$$It is not in general positive-definite, because e.g. if the $X_i$ are linearly dependent some nonzero choice of the vector $v$ ensures $\operatorname{Var}(v_iX_i)=0$. But if no linear combination of the variables is constant, the covariance matrix is positive-definite.