Multivariable calculus for data set

Question

My question I am trying to attempt is below as follows:
Consider a data set ($x_1,y_1$), ($x_2, y_2$),..., ($x_n,y_n$), for which we want to find the line of best fit. The "method of least-squares assume the line of best fit has the form $y = ax +b$, and minimize the square of the error, defined as $$E(a,b) = \frac 1n \sum_i^n(y_i-y(x_i))^2$$
1. Let the mean of a sequence be $\bar z = \frac 1n \sum^n_i_=_1 z_i$, hence show that $$E(a,b) = \bar y^2 - 2a \overline x \overline y - 2b \bar y + a^2 \bar x^2 + 2ab \bar x + b^2$$ 2. Show that the only critical point of E(a,b) is $$a = \frac {\bar x . \bar y - \overline x\overline y}{(\bar x)^2 - \bar x^2}, b = \frac {-\bar x^2 . \bar y + \bar x .\overline x\overline y}{(\bar x)^2 - \bar x^2}$$ 3. Given that $\bar x^2 > (\bar x)^2$, show that the critical point is a local minimum. 4. The following table shows the world population y (in billions) for five different years.
Let x = 3 represent the year 2003. $$ \begin{array}{c|c|c|c|c} Year,2000 + 3 & 2003 & 2005 & 2007 & 2009 & 2011 \\ \hline Population,y & 6.3 & 6.5 & 6.6 & 6.8 & 6.9\\ \end{array} $$ Use the method of least-squares to calculate a and b, and predict the population for the year 2020.

Can somebody please give me pointers on how to do each of the questions? I'm not quite sure where to begin with each question, I have a vague idea for question 2 but the rest of them I'm not sure.

Answer 1

OK, just to give you some intuition. For problem no. 1, expand the expression under the square:

$$\sum_{i=1}^{n}\left(y_{i}-y(x_{i})\right)^{n}=\sum_{i=1}^{n}\left(y_{i}-ax_{i}-b\right)^{2}=\sum_{i=1}^{n}\left(b^{2}+2abx_{i}+a^{2}x_{i}^{2}-2by_{i}-2ax_{i}y_{i}+y^{2}\right)$$ Hence: $$E(a,b)=\frac{1}{n}\sum_{i=1}^{n}b^{2}+\frac{1}{n}\sum_{i=1}^{n}2abx_{i}+\frac{1}{n}\sum_{i=1}^{n}a^{2}x_{i}^{2}-\frac{1}{n}\sum_{i=1}^{n}2by_{i}-\frac{1}{n}\sum_{i=1}^{n}2ax_{i}y_{i}+\frac{1}{n}\sum_{i=1}^{n}y^{2}$$ Now, use the definition of mean given in the problem, use the fact that you can bring a constant in front of the series sign, and finally, use the fact that if $\alpha$ is a constant, $\sum_{i=1}^{n}\alpha=n\cdot\alpha$

EDIT

In order to solve problem no 2., you have to find the partial derivatives of $E(a,b)$ with respect to $a$ and $b$ and set them to zero:

$$\frac{\partial{}E(a,b)}{\partial{}a}=-2\bar{x}\bar{y}+2a\bar{x}^{2}+2b\bar{x}=0$$ and $$\frac{\partial{}E(a,b)}{\partial{}b}=-2\bar{y}+2a\bar{x}+2b=0$$ Solve the above system of equations for $a$ and $b$, and check with the answer provided.

EDIT2

Divide the first equation by $2$. You get $-\bar{x}\bar{y}+a\bar{x}^{2}+b\bar{x}=0$. From the second equation, as you wrote, you obtained $a=\left(\bar{y}-b\right)/\bar{x}$. Now plug it into that first equation. You get: $$-\bar{x}\bar{y}+\frac{\bar{y}-b}{\bar{x}}\cdot\bar{x}^{2}+b\bar{x}=0$$ which is: $$\frac{\bar{y}-b}{\bar{x}}\cdot\bar{x}^{2}+b\bar{x}=\bar{x}\bar{y}$$ Now multiply both sides by $\bar{x}$: $$\bar{y}\bar{x}^{2}-b\bar{x}^{2}+b\left(\bar{x}\right)^{2}=\bar{x}\cdot\bar{x}\bar{y}$$ Reorganizing the terms yields: $$b\left(\bar{x}\right)^{2}-b\bar{x}^{2}=\bar{x}\cdot\bar{x}\bar{y}-\bar{x}^{2}\bar{y}$$ Solving for $b$: $$b=\frac{\bar{x}\cdot\bar{x}\bar{y}-\bar{x}^{2}\bar{y}}{\left(\bar{x}\right)^{2}-\bar{x}^{2}}.$$ As you can see, the result is correct.