Least square method via calculus

Question

Let $f:R^n\to R$ be $f(x)=||Ax-b||$.

Prove that $x \in R^n$ where $f$ is minimum meets equations $A^TAx=A^T b$.

Any solutions? I have tried to count derivate from it but I could not get any result.

Answer 1

I'm going to assume that the norm you're using is the Euclidean norm.

Any minimiser of $f$ must also minimize $g$ where $$g(x) = \frac12 f(x)^2 = \frac12||Ax - b||_2^2 = \frac12(Ax - b)^\top (Ax -b).$$

Then $g'(x) = 0$ implies that $$A^\top(Ax - b) = 0$$ so $$A^\top Ax = A^\top b.$$