The notes and book I am currently using for my model building class are unclear, so to math stack exchange I go.
Here is my work so far:
Assuming $f_a(x) = ax$ and where $e_i = | f_a(x-i) - y_i |$
$e_1 = |-1| $, $e_2 = |a - 3|$, $e_3 = |3a - 8|$, $e_4 = |6a - 16|$
Using criteria: $\sqrt{ e_1^2 + \dots + e_n^2 }$
$$F(a) = \sum_{i=0}^4 e_i^2 = -1^2+(a-3)^2+(3a-8)^2+(6a-16)^2 = 46a^2 - 246a + 330$$
Take derivative of F with respect to a to find critical point.
$$\frac{dF}{da} = F'(a) = 92a - 246 = 0 \rightarrow a = \frac{123}{46} \approx 2.67$$
We will call this solution $\hat{a} \approx 2.67$
Moving forward we have,
$$\sqrt{ F(\hat{a}) } = \sqrt{46(2.67)^2 - 246(2.67) + 330} \approx 1.053$$
Here is where I start to get lost. Using criteria: $e_1 + \dots + e_n$
$ |-1| + |a - 3| + |3a - 8| + |6a - 16|$
Using from what I gather from my notes you take the positive and negative of all of errors, and place them on a number line.
$$-1 + (a-3) + (3a-8) + (6a-16) \quad \quad (1a)$$
$$1 - (a-3) - (3a-8) - (6a-16) \quad \quad (2a)$$
Simplifying $(1a)$ and $(2a)$, respectively brings the results
$$10a - 28 \quad \quad (1b)$$
$$-10a + 28 \quad \quad (2b)$$
From here I do not know what to do further, and I do not know how to complete the max criteria given above in the question.
This is not homework, but a study guide, and any help would be appreciated.
Merit functions and results
Let's first look at the merit functions to be minimized.
In the $l_{2}$ norm, the merit function is $$ M_{2}(a) = \sum_{k=1}^{4} \left( a x_{k} - y_{k} \right)^{2}, $$ which is plotted below.
In the $l_{1}$ norm, the merit function is $$ M_{1}(a) = \sum_{k=1}^{4} \left| a x_{k} - y_{k} \right|, $$ which is plotted below.
Finally we have the $l_{\infty}$ norm, the merit function is $$ M_{\infty}(a) = \max_{k=1,4} \left| a x_{k} - y_{k} \right|, $$ also is plotted below. For this plot, the colors tags the index of the dominant error term: black represents $r_{1}(a)$, blue $r_{4}(a)$.
Derivations
$l_{2}$
An advantage for the $l_{2}$ norm is now apparent: the derivative is continuous and we may use the calculus to find the minimum.
The requirement $$ \frac{\partial } {\partial a} M_{2} = -2 \sum_{k=1}^{4} \left( a x_{k} - y_{k} \right) = 0 $$ leads to $$ a_{2} = \frac{ \sum_{k=1}^{4} x_{k} y_{k} } { \sum_{k=1}^{4} x_{k}^{2} } = \frac{123}{46} \approx 2.67391. $$ The total error in the $2-$norm is $r_{2} = \sqrt{1.11}$.
Another tool is the pseudoinverse. The linear system is $$ a_{2} x = y $$ which has solution $$ a_{2} = x^{\dagger} y $$ where the pseudoinverse of a nonzero vector is $$ x^{\dagger} = \frac{x^{*}}{x^{*}x}. $$
Another choice is to use the normal equations, which is equivalent to the calculus solution: $$ x^{*}x a_{2} = x^{*} y , \qquad \Rightarrow \qquad a_{2} = \frac{x^{*} y} {x^{*}x}. $$
$l_{1}$
The absolute value is the archetype of continuous functions with discontinuous derivatives, and the absolute value is at the heart of the $l_{1}$ merit function. The method used here is sample and compare. Pick $a$, compute $r(a)$, check for minimum, increment $a$. The result is $$ a_{1} = \frac{8}{3}, \qquad r_{1} = \frac{4}{3}. $$
$l_{\infty}$
The same solution method was used as for the $l_{1}$ norm. A new detail is that because the error comes from a single term, we wanted to tag the source term. The result is the continuum of answers $$ a_{\infty} = \left[\frac{5}{2}, \frac{17}{6}\right], \qquad r_{\infty} = 1 $$ The answer shows that the first point provides a minimum error of $1$ because the solution must go through the origin. As the slope moves below $5/2$ or above $17/6$, the error will be dominated by the 4th point.