Why can we linearize the exponential regression?

Question

When we are using observed data $(x_1,y_1)\ldots(x_m,y_m)$ for the exponential model $$y(x)=a_1\mathrm{e}^{a_2x}~(a_1>0),$$ it is natural to think about the linearized model $$\ln y=\ln a_1+a_2x.$$

It is not hard to understand this approach, but my question is, how can we prove rigorously that we are obtaining the same results? I.e., the following two optimization problems for $a_1$ and $a_2$ $$\text{minimize}\sum_{k=1}^m(a_1\mathrm{e}^{a_2 x_k}-y_k)^2$$ and $$\text{minimize}\sum_{k=1}^m(\ln a_1+a_2 x_k-\ln y_k)^2$$ yield the same $a_1$, $a_2$?

My attempts using multivariate calculus failed, and I can only say that since the problems are "equivalent" with unique solution, the answer should be unique. Is that acceptable?

Answer 1

You will not obtain the same results at all beacause the residues are totally different. In the first case, they are $$r_i=\hat y_i-y_i$$ while in the second case, they are $$r_i=\log(\hat y_i)-\log( y_i)$$

If you look at my answer to this question, I explain that the first case corresponds to the usual minimization of the sum of the squares of absolute errors while the second corresponds more or less to the minimization of the sum of the squares of relative errors which is totally different.

Answer 2

There is a reason why your attempts failed! If you apply linear least squares to

$$\ln y=\ln a_1+a_2x$$

you will not achieve the least squares solution to

$$ y(x)=a_1\mathrm{e}^{a_2x} $$

but in the vast majority of cases you will find parameters that are EXTREMELY close to the solution. Usually we use the linear least squares solution as an initial guess for something like Levenberg-Marquardt or Newton's method which we will apply to the problem

$$ \text{minimize}\sum_{k=1}^m(a_1\mathrm{e}^{a_2 x_k}-y_k)^2$$

Answer 3

If there is no scatter on the data $(x_1,y_1)\ldots(x_k, ,y_k)\ldots(x_m,y_m)$ ,

that is if $\quad y_k=a_1\mathrm{e}^{a_2x_k}\quad$ for all $k$ ,

the regression calculus for $a_1 , a_2$ gives exactly the values of $a_1 , a_2$ whatever the criteria of fitting is. $$ $$

The criteria of fitting can be

LMSE : $\text{minimize}\sum_{k=1}^m(a_1\mathrm{e}^{a_2 x_k}-y_k)^2$ ,

or LMSRE : $\text{minimize}\sum_{k=1}^m(\ln a_1+a_2 x_k-\ln y_k)^2$ ,

or LMAE : $\text{minimize}\sum_{k=1}^m\big|a_1\mathrm{e}^{a_2 x_k}-y_k\big|$ ,

or others.

This is obvious because every term of the sum is exactly nul. The sum is nul in all cases which is the minimum possible.

$$ $$

But if the data is scattered,

that is if $\quad y_k=a_1\mathrm{e}^{a_2x_k}+\epsilon_k\quad$ with small random values $\epsilon_k$ ,

the terms of the sum are not nul and the sum is not nul but depends on the choice of the criteria of fitting as shown above.

Nevertheless the values of $a_1 , a_2$ computed according to each criteria of fitting are close one to the other insofar the scatter is small.

So, don't be surprized to find slightly different results for $a_1 , a_2$ if you chose different criteria of fitting.

Answer 4

You can try the following.

Suppose your data has just two points $(x_1,y_1)=(1,1), (x_2,y_2) = (2,2)$. Now fix one of the parameters in both of the problems, say $ a_2 = 1 $. Use calculus to determine what the corresponding 'minimizing' $ a_1 $ in each of the problem will be. Do they match? This should be easy to check due to the small size of data.