Positive SemiDefinite Matrix. Theory vs Matlab

Question

For my problem, I am looking for optimal vector $q$ that minimizes mean squared error (MSE) that has a form $$MSE=q^{T}(R_{xx}-R_{yx}^{T}R_{yy}^{-1}R_{yx}^{})q^{}.\tag{1}$$

Since $MSE\geq0$, it turns out that $P=R_{xx}-R_{yx}^{T}R_{yy}^{-1}R_{yx}^{}$ is a positive semi-definite matrix. Note that $R_{xx}^{}, R_{yy}^{}$ are correlation matrices and $R_{yx}^{}$ is a cross-correlation matrix.

I know that solution to eq(1) comes from eigen value decomposition of $P$.

However when I try to implement in MATLAB, some of the eigen values of $P$ are coming out to be negative. Can someone please point out how should I go about this issue.

Answer 1

Since $P=R_{xx}-R_{yx}^{T}R_{yy}^{-1}R_{yx}^{}$ is the Schur complement of the block matrix $$ R:=\left[ \begin{matrix} R_{xx} & R_{yx}^T \\ R_{yx} & R_{yy} \end{matrix} \right] $$ (w.r.t. the block $R_{yy}$) the Schur complement lemma states that $P$ is p.d. iff $R$ is p.d..

So you definitely have an issue within MATLAB.

Answer 2

Aren't you simply seeing the effects of finite precision in floating-point numerics?

Simple MATLAB example. This should trivially be positive semidefinite

x = ones(10,1);
X = x*x';

but it is not in floating-point numerics

>> min(eig(X))
ans =
 -2.3078e-16