Projection from a full-rank PSD manifold to a lower-rank PSD manifold

Question

Let $\mathcal{S}_+(n,k) = \{X\in\mathbb{R}^{n\times n}: X=X^\top \succeq 0, \text{rank}(X)=k\}$ be the manifold of positive semidefinite (PSD) matrices. I was wondering how to project $X\in \mathcal{S}_+(n,n)$ in a full-rank PSD manifold to a lower-rank PSD manifold $Y\in \mathcal{S}_+(n,k), (k<n)$. Can this projection be obtained by solving

$\arg\min_{Y} ||X-Y||_{\mathtt{F}}, \text{ s.t. rank}(Y)=k $

where $||\cdot||_\mathtt{F}$ is the Frobenius norm? My concern is, if the Frobenius norm is a correct formulation of the projection?

I am not quite familiar with the Riemann geometry, so this question may be stupid.

Thanks for the help!

Answer 1

One thing you can do is to use the singular value decomposition $S=UDU^{-1}$ with $U$ orthogonal and $D$ diagonal with positive decreasing values. Then replace $D$ by a diagonal matrix $D_k$ whose first $k$ elements in the diagonal are the same as those of $D$ and the others are zero. Then $S_k=UD_kU^{-1}$ can be defined as the projection.

This solution may not be unique in case of equal eigenvalues. I am pretty sure that it is equivalent to your argmin formulation. Note that this is not a linear projection.

Answer 2

That is correct. In general, for $n\times n$ matrix $X$ with singular value decomposition $U D V$, the matrix of rank $\le k$ closest to $X$ is $X_k= UD_k V$, where $D_k$ is obtained from $D$ by the smallest $n-k$ diagonal values to $0$. One sees that if $X$ is positive semidefinite, then $X_k$ is also positive semidefinite.

( the distance is given by the Frobenius norm, or any other unitarily invariant norm)

The relevan notion here is : low rank approximation