Could someone kindly suggest a method of solving the following constrained (equality and inequality) system of equations in the least squares fashion?
$$\min_x\frac 12\|Ax-b\|_2^2$$
such that
$$\begin{matrix}x_i + x_j = 1 \quad \text{for a certain }i,j\\0 < x_k < \tau_k \quad \text{where } k\ne i,j\end{matrix}$$
$\tau_k$ is the upper bound for $x_k$.
I would like to develop a solver for this in MATLAB. Would a gradient-descent based method a proper approach?
On
You can solve this problem using the projected gradient method (or an accelerated projected gradient method). Let $S$ be the set of all $x$ that satisfy the given constraints. The gradient of the objective function $f$ is $$ \nabla f(x) = A^T(Ax - b) $$ and the projected gradient iteration is $$ x^+ = P_S(x - t \nabla f(x)). $$ The function $P_S$ projects onto $S$. This projection step requires solving a linear algebra subproblem. (Note that the second set of constraints can be handled independently of the first set of constraints. If the linear constraints are described more explicitly, we can give more detail.) The projected gradient iteration will converge if the step size $t > 0$ is sufficiently small.
As other mentioned, you can easily use CVX or YALMIP together with a free solver (such as SDPT3 or Sedumi) to solve the problem. If you want to write your own solver, you should use a primal-dual approach to solve the problem. There are many papers for solving convex problems subject to linear equality constraints. See this fantastic book for more detail