Curve fitting: $a f_1(x)+b f_2(x)+c f_3(x)+d f_4(x)$

Question

I want to fit my data $y$ with a method such as $$a f_1(x)+b f_2(x)+c f_3(x)+d f_4(x)$$ However I don't know how to obtain coefficients $(a,b,c,d)$ of the known functions $(f_1(x),f_2(x),f_3(x),f_4(x))$. I don't know least squares very well. Therefore I want a matrix formula that I can get the coefficients for any data easily, without getting into learn least squares since I don't have time. Could you please help me?

Answer 1

Define a grid of $m$ values $x_i$ corresponding to your data $y_i$. Then you get an overdetermined system of linear equations:

$$\mathbf{Ax}=\mathbf{y}$$

where $\mathbf{A}$ is a $m\times 4$ matrix with elements $a_{ij}=f_j(x_i)$, $\mathbf{x}=[a,b,c,d]^T$ is your vector of unknowns, and the vector $\mathbf{y}$ contains your data. You can solve this system in a least squares sense. Depending on the software you use, you can either directly solve it like $$\tt{x=A \backslash b}$$ in Matlab or Octave. Otherwise you need to compute the pseudo-inverse and obtain the least squares solution like this:

$$\mathbf{x}=(\mathbf{A}^T\mathbf{A})^{-1}\mathbf{A}^T\mathbf{y}$$

Answer 2

Let's do least squares, as it gives you an equation as wished. $\def\R{\mathbb R}\def\Mat{\mathop{\mathrm{Mat}}\nolimits}$Let $A \in \Mat_{n,4}(\R)$ and $y \in \R^n$ be given by $$ A = \begin{pmatrix} f_1(x_1) & f_2(x_1) & f_3(x_1) & f_4(x_1) \\ & &\vdots & \\ f_1(x_n) & f_2(x_n) & f_3(x_n) & f_4(x_n)\end{pmatrix}, y =\begin{pmatrix} y_1 \\ \vdots \\ y_n\end{pmatrix} $$ where $(x_i, y_i)$, $1 \le i \le n$ are your data. We are asked to minimize $$ A \def\x{\begin{pmatrix} a \\b\\c\\d\end{pmatrix}}\x - y $$ The this is - supposing that $A$ has full rank - exactly the case this residual in orthogonal to the image of $A$, that is, iff $$ A^tA \x = A^t y, $$ so $$ \x = (A^tA)^{-1}A^ty. $$