"Rectangular" Cholesky decomposition of lower dimension

251 Views Asked by At

Given a symmetric PSD matrix $A \in \mathbb R^{n \times n}$, we can Cholesky-decompose it into $LL^T$, where $L \in \mathbb R^{n \times n}$ is lower triangular. However, we can also consider decompositions of the form $A \simeq X X^T$, where $X \in \mathbb R^{n \times m}$ and $m \neq n$.

I assume that by some rank argument, this is only possible to be solved exactly of $m \geq n$. However, what if I am interested in the following optimisation problem in $X \in \mathbb R^{n \times m}$

$$\text{minimise} \quad \| A - XX^T \|_2$$

where $m \ll n$? Is this some standard problem that has been studied? Can I use some kind of solver to solve this style of problem? I'm not well versed in SDP / cone programming, can this problem be phrased in terms of those style of problems?

1

There are 1 best solutions below

4
On BEST ANSWER

The Eckart-Young-Mirsky theorem tells us that the best rank $r$ approximation to a possibly non-symmetric matrix $A$ in the 2-norm (also in the Frobenius norm) is given by the first $k$ singular values and singular vectors of the SVD of $A=U\Sigma V^{T}$ as

$A=\sum_{i=1}^{k} \sigma_{i} U_{i}V_{i}^{T}$.

When $A$ is symmetric and positive semidefinite, this simplifies to

$A=\sum_{i=1}^{k} \sigma_{i}U_{i}U_{i}^{T}$

and this can be written as

$A=XX^{T}$

where

$X=\left[ \begin{array}{cccc} \sqrt{\sigma_{1}}U_{1} & \sqrt{\sigma_{2}}U_{2} & \ldots & \sqrt{\sigma_{k}}U_{k} \end{array} \right] $

There's no need for SDP to solve this problem.