I'm trying to find the optimal solutions for a multivariable exponential function. The function is in the form as:
$y_{i}=\left ( \beta_{1}x_{i1}+\beta_{2}x_{i2} +\beta_{i3}x_{3}\right )e^{\beta_{4}x_{i4}}$
where $\beta_{1-4}$ are to-optimize parameters. I try to solve this problem by lease square as:
$argmin {\sum (Y - y_{i})}^2$
But I'm not sure how to update the gradient for this function. Linear functions are pretty easy. I tried optimize.leaseq in scipy and it works. But I want to know how internally it calculates. Thanks,
This is too long for a comment.
Assuming that you have data points $(x_{1i},x_{2i},x_{3i},x_{4i},y_i)$ and the model $$y=\left ( \beta_{1}x_{1}+\beta_{2}x_{2} +\beta_{3}x_{3}\right )e^{\beta_{4}x_{4}}$$ I should fix $\beta_{4}$ at a given value and define $z_i=e^{\beta_{4}x_{4i}}$ making the model linear with respect to the remaining parameters.
So, for a given value of $\beta_{4}$, we can get the other $( \beta_{1},\beta_{2},\beta_{3})$ and the corresponding objective function (whatever it could be). Now, try different values of $\beta_{4}$ until you see an area where a minimum seems to happen for the objective function. At this point, you have all elements for starting the full process.
I must confess that I am thinking more about nonlinear regression than about any other method.