Using the least squares method for problems with two independent variables

317 Views Asked by At

This may be quite a specific question and I apologise however I have struggled to find any information regarding a method.

I have 6 given $P_i$ values and 6 given $E_i$ and $F_i$ values and I want to find the "best" values of $a$, $b$, and $c$ which give the minimum of $\sum_{i=1}^6 (P_i - a - b E_i - c F_i)^2$.

I have been trying to use the 'least squares' method by plugging in all data in the model to get an over-determined linear system of equations but I can't get the data into matrix form, i.e. $Ax=b$ to solve $A^TAx=A^Tb$.

Any guidance would be appreciated, thanks.

1

There are 1 best solutions below

0
On

The symmetric matrix $A$ you form has row and column indices corresponding to the terms in the fit; in this case, the coefficients of $E$, $F$, and $1$, or $b$, $c$ and $a$. The column matrix $b$ also has rows with these same indices. The specific formula is $$ A = \pmatrix{ \sum_i 1\cdot 1 = n & \sum_i 1 \cdot E_i & \sum_i 1 \cdot F_i \\ \sum_i E_i\cdot 1 = \sum_i E_i & \sum_i E_i \cdot E_i = \sum_i E_i^2 & \sum_i E_i \cdot F_i \\ \sum_i F_i\cdot 1 = \sum_i F_i & \sum_i F_i \cdot E_i & \sum_i F_i^2 \\ } $$ and the column vector on the right is $$ b = \pmatrix{ \sum_i P_i \\ \sum_i E_iP_i\\ \sum_i F_iP_i} $$ Done in this way, the equations are not overdetermined, and unless you are unlucky enough to have $\mbox{det } A = 0$ there is a unique best-fit solution for $x = (a,b,c)^T$, obtained by solving $$ Ax = b $$