I am trying to make a prediction model based on a system of linear equations: $A\vec{x}=\vec{b}$, where $\vec{x}$ ($m\times1$) is my learning parameters, $A (m\times n)$ and $\vec{b}$ $(m\times1)$ are known measurements from the training experiment. The system is overdetermined ($m>n$). To do this, I minimize $||A\vec{x}-\vec{b}||_2$. The best way I found so far is truncated SVD: making SVD of $A$, getting rid of small singular values and finding solution as $\vec{x}=V\Sigma_{truncated}U^T\vec{b}$. The optimum number of truncation is determined by testing some dataset and minimizing RMSE. My best SVD solution gives me some components of $\vec{x}$ pivoting around zero, something like this: $\vec{x}=||0; 0.1; 0.3; -0.4; 0.2; -0.1; -0.3; 0.4; -0.25...||$ (totally I have 36 values)
So far so good, but I have to constrain my solutions $\vec{x}$ to only positive values. I tried to use CVX toolbox in Matlab for that, but with no luck: the solution simply ends up in $\vec{x}$ with all zero components except one ($\vec{x}=||0; 0; 0;...1||$). RMSE of such solution is several orders of magnitude higher than that of the optimum SVD. For some prediciting sense of my model, I would expect to get at least several positive components in my solution (the most important characteristics of the system). Here is what I am doing with CVX in Matlab. Unfortunately, I cannot give you the matrix $A$ and $\vec{b}$, since it is confidential.
cvx_begin
variable x(n)
minimize norm(A*x-b)
subject to
x >= 0
cvx_end
Does the experts know a better optimization tool?