Recently I have been working on a project involving stock returns and portfolio efficiency. What I thought was a coding error, turned out to be me not understanding matrix math very well.
Long story short, the R function I am using needs a positive definite covariance matrix. I do not have any problems as long as the matrix I am using to calculate the covariance matrix has fewer columns than rows. Once the number of columns equals or exceeds the number of rows, then my covariance matrix is not longer positive definite and my code fails.
A small-scale example. Let matrix A be my original matrix I am using to calculate matrix B, my covariance matrix. Say A is 3x2. B turns out to be positive definite, and my code works fine. Once I add another column of data to A, making it 3x3, B is no longer positive definite. This same situation occurs no matter the size. 5x2, 8x5, 16x8, 39x38 all produce PD B matrices. But once the number of columns exceeds the number of rows (5x5, 8x11, 39x40), B is no longer PD.
Is this just a matrix thing that I never learned? Is positive definiteness dependent on the matrix dimensions?
Here is the original thread that spurred this question: https://stackoverflow.com/questions/31007182/portfolio-frontier-not-working-for-large-dataset
If I was unclear please let me know. Edit: Here are a few numerical examples.
[-.0108, .0469
.0050, .0193
-.0059, -.0005]
This is an example of my original matrix, A. This leads to a PD B matrix of
[.00006, -.00007
-.00007, .0005]
If A is
[-.0108, .0469, .0190, .0080
.0050, .0193, .0150, .0177
-.0059, -.0005, .0091, -.0023]
Then B will be not PD.
Of course, assuming X has more columns than rows, if $\mathbb{E}X=0$ then $\Sigma=X^TX$. To check positive semi-definiteness, we evaluate $$y^T\Sigma y=y^TX^TXy=\left\|Xy\right\|_2^2\geq0$$ which must in fact be zero some of the time since $X$ has more columns than rows.
As it turns out mean subtraction actually reduces the rank of your data matrix $X$ by one (proof not forthcoming), so even if $X$ is square, mean subtraction makes it singular and thus it will have non-trivial kernel. Thus the number of rows of your data matrix X must be strictly greater than the number of columns for the covariance matrix to be strictly positive definite.