How to apply the method least squares polynomial of single degree?

Question

Now I am making Almon model. Lag is 3, and polynomial of 2 degree, so I have following linear regression equation $y_{t}$ = $a$ + $c_{0}$$z_{0}$+ $c_{1}$$z_{1}$+$c_{2}$$z_{2}$. I have a list of $y_{t}$, $z_{0}$, $z_{1}$, $z_{2}$ values, how can I apply method least squares to calculate $c$'s?

This method pointed in a book I am reading. I only know how to use this method with models like $P_{m}(x) = a + a_{1}x + a_{2}x^2 + a_{3}x^3+...+a_{m}x^m$

Answer 1

We can rewrite the regression equations slightly as $$ z_0 c_0 + z_1 c_1 + z_2 c_2 = y_t - a $$ then arrange them as linear system $$ A x = b $$ with $$ A = (z_0, z_1, z_2) \\ x = (c_0, c_1, c_2)^t \\ b = (y_t-a) $$ where the $z_i$ and $y_t-a$ are column vectors each, having $k$ components if your data list has $k$ lines of such data. $k \ge 3$ would be needed.

Then extend to $$ A^t A x = A^t b $$ then $$ x = (A^t A)^{-1} A^t b \quad (*) $$ is a least square approximation, minimizing $$ A x - b $$ in the Euclidean norm.

Example Calculation

Entering some matrix $A$ and vector $b$ for four lines of data:

octave:1> A = [1, 2, 3; 2, 3, 1; 3, 3, 2; 2, 2, 3] 
A =

   1   2   3
   2   3   1
   3   3   2
   2   2   3

octave:2> b = [1;2;2;1]
b =

   1
   2
   2
   1

Calculating the transposed matrix $A^t$:

octave:3> A'
ans =

   1   2   3   2
   2   3   3   2
   3   1   2   3

Calculating the approximation $x$:

octave:4> x = inv(A'*A) * A' * b
x =

   0.079755
   0.674847
  -0.153374

Checking the error vector and its length:

octave:5> e = A*x - b
e =

  -0.030675
   0.030675
  -0.042945
   0.049080

octave:6> el = norm(e)
el =  0.078326

Answer 2

Look up regression analysis, particularly the general linear model (in several variables). There is a huge literature on the subject, including estimating errors of the coefficients.