I would like to solve the following problem with singular value decomposition. We have $x_{1},...,x_{m}$ vectors of $\mathbb{R}^{d}$. But $d$ is too high for your computer. So Let's find a linear application $W : \mathbb{R}^{d} \rightarrow \mathbb{R}^{n}$ with $n < d$ such that $\tilde{x} = Wx$ is the compression of $x$. And note $U : \mathbb{R}^{n} \rightarrow \mathbb{R}^{d}$ the decompression of $\tilde{x}$.
The question in our mind: is $x \approx \tilde{x}$ ?
Mathematically the question is to find two matrices $W \in M_{n,d}(\mathbb{R})$, $U \in M_{d,n}(\mathbb{R})$ such that $$ \sum_{i=1}^{m} \| x_{i} - UWx_{i} \|_{2}^{2} $$ is minimal.
What did I try ?
I try to use SVD, let's $X = (x_{1},...,x_{m}) \in M_{d,m}(\mathbb{R})$ it's SVD is $\sum_{i=1}^{r} \sigma_{i}u_{i}v_{i}^{T}$ with $r$ the rank of $X$ and $u_{i}, v_{i}$ two orthonormal famillies of $\mathbb{R}^{d}$ and $\mathbb{R}^{m}$. We have $$ \sum_{i=1}^{m} \| x_{i} - UWx_{i} \|_{2}^{2} = \| X - UW X \|^{2}_{F} $$ with $F$ the Frobenius norm.
Now we see that the rank of $U$ is $\le n$ and we know that $$ \text{argmin}_{B ; \text{ rank }(B) \le k}{\| X - B \|_{F}^{2} } = \sum_{i=1}^{k} \sigma_{i}u_{i}v_{i}^{T} $$ We can understand the problem is solved if we can find $U$ and $W$ such that $UW = \sum_{i=1}^{n} u_{i}u_{i}^{T}$.
And I can't go any further.
Thank you for your help.