Hi I wanted to know how am I suppose to approach this problem. The section we are working on is 4.3 of gilbert strang book. It is on Least squares approximation but there is no example on how to solve this.
Find the best line $At + B$ approximating the data set $b$ at the times $t = (0, 1, 2, 3, 4)$.
(a) $b = (−1, −1, 2, 0, 0)$.
(b) $b = (−1, 0, 2, 0, 0)$.
Let me solve your first exercise for you. The second one should be completely analogous, so I will leave that one to you.
$\textbf{First possibility}$: A first possibility to solve this question is by using the definition of 'least squares'. We want to minimize the errors which are made by approximating the given points by a line. Given the points $(t_i, f_i)$, define the error made by using the linear approximation by $r_i$. We find: \begin{equation} r_i = At_i + B - f_i \quad (\ast) \end{equation} for every $i \in \{0,1, \ldots, 4\}$. Now we want to minimize the sum of the squares of these errors (we consider the squares of the errors in order to avoid that positive errors cancel negative errors!). So we want to minimize the following function: $$\varphi(A,B) = \sum_{i = 0}^4(At_i + B - f_i)^2.$$ In order to find this minimum, we need to compute the partial derivatives with respect to $A,B$ and equalize them to 0. This gives us the following system of equations: $$\begin{cases} \sum_{i = 0}^4(At_i + B - f_i)t_i = 0\\ \sum_{i = 0}^4(At_i + B - f_i) = 0 \end{cases}.$$ This can be rewritten: $$\begin{cases} \sum_{i = 0}^4(At_i^2 + Bt_i -f_it_i) = 0\\ A\sum_{i = 0}^4t_i + 5B = \sum_{i = 0}^4f_i \end{cases},$$ hence we find $$\begin{cases} A\sum_{i = 0}^4t_i^2 + B\sum_{i = 0}^4t_i = \sum_{i = 0}^4f_it_i\\ A\sum_{i = 0}^4t_i + 5B = \sum_{i = 0}^4f_i \end{cases},$$
Let us fill in your points. This gives us $$\begin{cases} 30A + 10B = 3\\ 10A + 5B = 0 \end{cases}$$ (Check this yourself). Solving for $(A,B)$ gives $(A,B) = (0.3, -0.6)$, showing that the line has equation $l \leftrightarrow 0.3x - 0.6 = y$.
$\textbf{REMARK}$: We could have done this more generaly for $N$ points. In order to do this, in equation $(\ast)$ you change $4$ into $N - 1$ (or you completely change the sum to make it go from $i = 1$ to $i = N$, which is possible since $i$ is just a dummy variable, so we can shift its range). You then also change this in all further equations following from this equation!
$\textbf{Second possibility}$: We want to find a polynomial of degree 1 to approximate the given data points. Let me first denote the given information more generaly: you are given five points $\{(t_0, f_0), \ldots, (t_4, f_4)\}$, where I denoted the vector $(t_0, \ldots, t_4)$ to represent the times you are given and the vector $(f_0, \ldots, f_4)$ denotes the given vector $b$.
In the ideal case, we want to simultaniously solve the following equations for $i \in \{0,1, \ldots, 4\}$: $$A t_i + B = f_i,$$ and this would result in solving the following linear system for $A,B$ (filling in the information you are given in exercise (a): $$\begin{pmatrix} 1 & 0\\ 1 & 1\\ 1 & 2 \\ 1 & 3 \\ 1 & 4 \end{pmatrix} \cdot \begin{pmatrix} B \\ A \end{pmatrix} = \begin{pmatrix} -1 \\ -1\\ 2\\ 0\\ 0 \end{pmatrix}.$$
I will denote this by $C \cdot x = F$.
However, we are given $5$ equations with only $2$ variables, so it is very unlikely that this system will have a solution. How to solve this problem? I am quite sure you have seen 'the normal equations' and this is the general way to solve a (discrete) least squares problem.
In stead of solving $Cx = F$, we solve $C^tCx = C^tF$. Note that $C^tC$ is a square $2 \times 2$ matrix and $C^tF$ is a $2 \times 1$ column vector. Doing this, we find the following system of equations: $$ \begin{pmatrix} 5 & 10\\ 10 & 30 \end{pmatrix}\cdot \begin{pmatrix} B \\ A \end{pmatrix} = \begin{pmatrix} 0\\ 3 \end{pmatrix},$$ and this system is the same system given in the first possibility to solve this question (well, the rows are switched, but this does not matter). Therefore, we will find the same solution.