The smallest error vector (e1,e2,e3,e4) is perpendicular to (.1,1,1,1), the first column of A. Least squares balances out the four errors, and they add to zero.
The last sentence is confusing as heck. I can't wrap my head around it..The author just threw this one out there at the end with no explanation. What is it referring to that the least squares balance out the errors? And they add to zero? What is 'they'? Least squares wouldn't add to 0.. And how is this tied to the previous sentence that the error vector is perpendicular to A's first column?
If someone understands this, please explain what the author wants to convery here. Thanks.
Last paragraph on page 7:
https://www.scribd.com/document/110574518/Strang-Least-Squares
(Starting from a little earlier to make a self-contained answer...)
If the point $(t_4, b_4)$ is not on the parabola through the first three points, then there is no parabola passing through all four points. Therefore, we cannot solve $A \mathbf{x} = \mathbf{b}$ and must instead project onto the (sub-)space of parabolas via $A^\mathrm{T} A \mathbf{\widehat{x}} = A^\mathrm{T}\mathbf{b}$. This projection is perpendicular to the space of parabolas, so the error vector (displacement vector from the starting point to the space of parabolas) is perpendicular to that space, that is, it is perpendicular to each member of a basis for that space. The first column of $A$, $\begin{pmatrix}1\\1\\1\\1\end{pmatrix}$ is one of the three column vectors of $A$, and those three column vectors are linearly independent, so they form a basis of the space of parabolas. Therefore, the error vector is perpendicular to $\begin{pmatrix}1\\1\\1\\1\end{pmatrix}$. Writing the error vector as $(e_1, e_2, e_3, e_4)$, perpendicularity forces $$ (e_1, e_2, e_3, e_4) \cdot \begin{pmatrix}1\\1\\1\\1\end{pmatrix} = e_1 + e_2 + e_3 + e_4 = 0 \text{.} $$ So even though we only know about this projection somewhat abstractly (even though we have $A$ and could compute $A^\mathrm{T} A$ easily enough), one thing we know is that the sum of the four error components is zero -- that is, the projection is constrained to only produce displacements whose components sum to zero.
We also know, from perpendicularity with the second and third columns of $A$, respectively, $e_2 + 3 e_3 + t_4 e_4 = 0$ and $e_2 + 4 e_3 + t_4^2 e_4 = 0$, but those aren't so easily expressed in words.