Consider the following problem: $$minimize\ ||Ax-b||^2,$$ where $A$ is an $m\times n$ matrix and $b$ is a vector of dimension $m$.
What would be a necessary condition of optimality?
First off I don't understand how is possible to minimize this $\ ||Ax-b||^2,$ I haven't seen problems stated like that, the usual style is min $f(x)\ s.t. ax<b, cx+e<d,$ etc.
Could someone light me up please? What should I do in this cases?
Hint:
with the usual norm you have a function $$ ||Ax-b||^2=\sum_{j=1}^m \left[\left(\sum_{i=1}^n A_{ji}x_i\right)-b_j \right]^2=f(x_1,x_2,\cdots,x_n) $$
do you know how to minimize a function of $n$ variables?