I came across a very common multi-modal robustness problem.
I have a simple linear equation set $Ax=b$ that I need to solve.
Obviously, SVD can be used for a least-squares solution for $x$.
However, the rows of $A_{N \times M}$ and $b$ come from $k$ different sources ($k \ll N$), so that there is not just one $x$ which is the solution but $k$ true $x$s.
In fact, if I could cluster/segment my data sources for each source separately the solution would be much more precise with a separate $x$ for each such cluster.
Is there a robust algorithm (reasonably fast would be nice too) for solving for such multiple $x$ and $k$? Maybe from the low-rank/sparse matrix decomposition literature? I am not familiar with the exact terms to search for.
What if $k$ is known?
What if $k$ is not known?
Any reference would be greatly appreciated!