Incremental Approach to Solve Positive Least Square Problem

Question

Is there any incremental (approximate) solution for the following positive least squares problem:

$$\min_x \|Ax-b\|^2\qquad \textrm{s.t.}\qquad x_i> 0,~b_1=1,~b_{i>1}=0$$

Answer 1

The problem as written above isn't convex.
If you relax the constraint into $ x \succeq 0 $ then the problem becomes a Convex Problem.

Now you can either solve it using Quadratic Programming Solver or just use the Gradient Descent Method where at each iteration you project the solution onto the non negative orthant:

$$ \begin{align*} {x}^{k + 1} & = {x}^{k} + \alpha {A}^{T} \left( A x - b \right) \\ {x}^{k + 2} & = \max \left\{ {x}^{k + 1}, 0 \right\} \end{align*} $$