My problem:
The point $(\bar x, \bar y)$ is the center of mass for the collection of points in Exercise 7. Show that the least squares line must pass through the center of mass. [Hint: Use a change of variables $z = x - \bar x$ to translate the problem so that the new independent variable has mean 0.]
I have already solved Exercise 7:
Given a collection of points $(x_1, y_1), (x_2, y_2), \ldots, (x_n, y_n)$, let $\mathbf x = (x_1, x_2, \ldots, x_n)^T$, $\mathbf y = (y_1, y_2, \ldots, y_n)^T$, $\bar x = \frac 1n \sum_1^n x_i$, $\bar y = \frac 1n \sum_1^n y_i$ and let $y = c_0 + c_1 y$ be the linear function that gives the best least squares fit to the points. Show that if $\bar x = 0$, then $c_0 = \bar y$ and $c_1 = \frac {\mathbf x^T \mathbf y}{\mathbf x^T \mathbf x}$.
It is obvious that if $x = \bar x$ then $y = c_0 + c_1x = \bar y + 0 = \bar y$, however the hint suggests that the problem should be solved in another way.
Edit I have found an answer. It makes use of the following theorem:
If A is an m x n matrix of rank n, the normal equations $ A^T A \mathbf x = A^T \mathbf b$ have a unique solution $ \hat {\mathbf x} = (A^TA)^{-1}A^T \mathbf b$ and $ \hat {\mathbf x} $ is the unique least squares solution of the system $ A \mathbf x = \mathbf b $.
Now let $ \hat {\mathbf x} = \mathbf c = (c_0, c_1)^T, A = \begin{pmatrix}1 & \cdots & 1 \\x_1 & \cdots & x_n \\\end{pmatrix}, \mathbf b = \mathbf y = (y_1, \ldots, y_n)^T $ such that $c = (A^TA)^{-1}A^Ty$, then $$\begin{pmatrix}c_0\\c_1\\\end{pmatrix} = \begin{pmatrix}n & \sum x_i\\\sum x_i & \sum x_i^2\\\end{pmatrix}^{-1} \begin{pmatrix}\sum y_i\\\sum x_iy_i\\\end{pmatrix} $$ which gives values for $c_0$ and $c_1$. These values should be used in the formula $c_1x + c_0$, which, together with $ x = \bar x = \frac 1n \sum x_i$, indeed results in $ \bar y $.
To show that the least squares line must pass through the center of mass, one does not need to invert the Gram matrix $A^TA.$
Instead, we start from $$ \begin{pmatrix} n & \sum_{i=1}^{n} x_{i} \\ \sum_{i=1}^{n} x_{i} & \sum_{i=1}^{n} x_{i}^{2} \end{pmatrix} \begin{pmatrix} c_{0} \\ c_{1} \end{pmatrix} = \begin{pmatrix} \sum_{i=1}^{n} y_{i} \\ \sum_{i=1}^{n} x_{i} y_{i} \end{pmatrix}, $$ which is equivalent to $A^{T}Ac = A^{T}y$ and is derivable from geometry. Next, we focus on the first equation in the system of linear equations, which is also known as the normal equations. Namely, we rewrite the first equation: $$nc_{0} + (\sum_{i=1}^{n} x_{i})c_{1} = \sum_{i=1}^{n} y_{i}$$ in terms of the center of mass entries: $$nc_{0} + n\bar{x}c_{1} = n\bar{y}.$$ After dividing by $n,$ we obtain $c_{0} + \bar{x}c_{1} = \bar{y},$ which is the desired conclusion.