least squares using exponential model

Question

I'm trying to fit values from this model $$y(x)=ae^{−bx}+c$$ where a, b and c are 3 different parameters that I want to find with least squares. So using least squares I want to find the value of a, b and c using this formula

$$\mathrm{argmin}\sum_i \left( ae^{−bx_i}+c - y_i \right)^2$$. This leads me to 3 non linear equations after partial derivation of the previous one with respect a, b and c. Surely there are already the general formula for this case but I cannot find them. Can anybody help me?

Answer 1

This is an example of non-linear regression. The softwares solve this kind of problems thanks to some algoritms with iterative methods, starting from guessed values of the parameters.

http://mathworld.wolfram.com/NonlinearLeastSquaresFitting.html

I would mention a straigtforward method (not iterative, no guessed values needed) :

https://fr.scribd.com/doc/14674814/Regressions-et-equations-integrales

No need to read the explanations in French. In case of $$y(x)=a+b e^{cx}$$ see page 17, just apply the equations which are are very simple and understandable in any language.

A rough translation is added below :

Answer 2

The problem is difficult because the model is nonlinear $\cdots$ because of parameter $b$. Just suppose you fix it at a given value; then the model becomes linear and you can compute the values of $a(b)$ and $c(b)$ as well as the value of the objective function $\Phi(b)$ you want to minimize.

So, run different values of $b$ until you notice that the function $\Phi(b)$ went through a minimum. At this point, you then have good estimates of the three parameters and you can start a nonlinear procedure. If you do not want, you can polish the solution using Newton-Raphson method for three equations (the derivatives) and the unknowns $(a,b,c)$.

This is a method I commonly use when the model is nonlinear with respect to a small number of parameters (if more than one, generate the value of the function over a grid). This works quite well.

Let me show you the last model for which I used this procedure.

$$P=\rho RT+(B_0RT-A_0-\frac{C_0}{T^2}+\frac{D_0}{T^3}-\frac{E_0}{T^4})\rho^2+(bRT-a-\frac dT)\rho^3+$$ $$\alpha(a+\frac dT)\rho^6+\frac{c \rho^2}{T^2}(1+\gamma \rho^2)e^{-\gamma \rho^2}$$ and the data are $(T_i,\rho_i,P_i)$ ($R$ being the ideal gas constant). The model is linear with respect to all parameters except $\gamma$. It worked very well.

This pocedure has (at least to me) the advantage of using the standard features of linear models. You can change it to a more mathematical one : use $\Phi'_a$ and $\Phi'_c$ to express everything in terms of $b$ and you are just facing the problem of solving $\Phi'_b=0$ for $b$.

Example for illustration

Uisng the data given on page 17 of the book by JJacquelin, I just performed a few calculations for specific values of $b$. The best three points found correspond to $b=1.50$ ($SSQ=0.183835$), $b=1.75$ ($SSQ=0.135040$), $b=2.00$ ($SSQ=0.164380$); for $b=1.75$, $a=0.328$ and $c=0.556$. Starting the nonlinear regression with these values makes the problem converging in three iterations and the final model is $$y=0.337706 +0.542859 e^{1.77478 x}$$ which corresponds to a sum of squares equal to $0.134660$.

Edit

The key problem is to know where to start searching for a reasonable value of parameter $b$. The general model being $$y=ae^{bx}+c$$ let us consider three data points indexed by $i,j,k$; so we can write $$\frac{y_i-y_j}{y_i-y_k}=\frac{e^{bx_i}-e^{bx_j}}{e^{bx_i}-e^{bx_k}}=\frac{1-e^{b(x_j-x_i)}}{1-e^{b(x_k-x_i)}}$$ Now, select $x_k=\frac{x_i+x_j}2$; this leads to $$\frac{y_k-y_j}{y_i-y_k}=e^{b\frac{x_j-x_i}2}$$ then $b$. Even if there is no such $x_k$ in the table, we can approximate it.

For the example from JJacquelin, I took for $x_i$ and $x_j$ the first and last values ($-0.99$ and $0.981$) and I used $y_k=0.911$ corresponding to the $x$ closest to $-0.0045$. This gives $b=1.7077$; reusing the end points gives $a=0.6063$ and $c=0.3062$ which are not bad at all as first guesses.