linear least square problems solved using LU decomposition

1.5k Views Asked by At

I have been given this datafile For which I have to solve Ax=b. In which A is a matrix, x a vector and b a vector.

The datafile consist of 2 vector one with the X-coordinates, and the other one with y-coordinates. I don't how i based from that shall create a A matrix, because that will make a A matrix on length of dataset) x 2?

2

There are 2 best solutions below

11
On BEST ANSWER

As science said, you have the following model:

$$ y_{i} = \beta_{0}+\beta_{1}x_{i}+\beta_{2}x_{i}^{2}+\varepsilon_{i}, \qquad i=1,2,\dots,n $$

For each of your data point you have this previous equation (there are 40 equations like that if I correctly understand the data set). You have 3 unknowns for 40 equations therefore you need to perform a regression. $\varepsilon$ is the error between the model and the data that needs to be minimized. You can re-write in a matrix notation:

$$ Y_{(n\times 1)} = X_{(n\times k)}A_{(k\times 1)} + \boldsymbol{\varepsilon}_{(n\times 1)} $$

where subscripts represent the dimension of the matrices. In your data set there is 40 observations and the model is quadratic, meaning that it has 3 parameters ($\beta$s). This translates the above regression in:

$$ Y_{(40\times 1)} = X_{(40\times 3)}A_{(3\times 1)} + \boldsymbol{\varepsilon}_{(40\times 1)} $$

where $X$ first column is filled of ones for the constant $\beta_{0}$, second column is the data and third column is filled of your squared data. And $A = (\beta_{0},\beta_{1},\beta_{2})^{T}$. To estimate the coefficient of matrix $A$ you generally end up with the following expression (which is the classical OLS) by minimizing the sum of squared difference between the data and the model ($\varepsilon$):

$$ \ \min_{\beta_{0},\beta_{1},\beta_{2}} \sum_{i=1}^{n=40}\varepsilon_{i}^{2} $$

and the result can be express in a matrix notation as (to keep things simple)

$$ A_{(3\times 1)} = (X_{(3\times 40)}^{T}X_{(40\times 3)})^{-1}X_{(3\times 40)}^{T}Y_{(40\times 1)} $$

and you can check that the matrix dimensions corresponds on each side of the equality side.

5
On

Here is how you get the matrix. You substitute the pairs $(x_i,y_i)$ in the model

$$ y = \beta_0+\beta_1 x +\beta_2 x^2 + \epsilon $$

to get the system of equations

$$ y_i = \beta_0+\beta_1 x_i +\beta_2 x_i^2 + \epsilon, \quad i=1,2,\dots, n $$

then you will get the matrix.