Which of the following lines offers the better fit to the given points in the least-squares?

Question

I have two similar exercises:

Which of the following lines, $y = 1 - x$ or $y = 4 - x$, offers the better fit to the points $(1,2),(2,1), (3,1)$ in the least-squares? Justify?

My thought process (haven't solved it yet) was to solve for $A^tAx=A^tb$ and find the line that best approximates it, and then for each point find the error... but now I see that doesn't make much sense. I might not have to solve that system because on this other exercise:

Determine and justify, without solving any system of equations, which line $y = -1 -x$ or $y = 1-x$ offers the better fit, in the least squares, the points $(-2,3),(-1,2),(0,1),(1,0)$.

How do I solve this? I remember vaguely some formula that involved a sum of the residue or error or something, and squares, but I don't remember it well.

Answer 1

The first one can be solved without computation at all.

Clearly $4-x$ is the better fit in the least-square sense. Maybe we'd rather have a formula. For each point, compute the square of the difference between the height of the point and the height of the line.

• $1-x$: $((1-\overbrace{1}^{x_1})-\overbrace{2}^{y_1})^2 + ((1-2)-1)^2 + ((1-3)-1)^2 = 4 + 4 + 9 = 17$.
• $4-x$: $((4-1)-2)^2 + ((4-2)-1)^2 + ((4-3)-1)^2 = 1 + 1 + 0 = 2$.

This computation confirms that $4-x$ is a better fit to the data in the least squares sense.

As a formula, the quantity to minimize is $$ \sum_{i=1}^n (f(x_i) - y_i)^2 \text{,} $$ where $\{(x_1, y_1), \dots, (x_n, y_n)\}$ is the set of points being fit and $f$ is (in these examples) the line.

The second one is even easier -- all the point lie on one of the lines.