Suppose I have the following formula:
$$y_t = \beta_0\sum_{i=0}^p w(\delta;i)x_{t-i}$$
Where $\displaystyle w(\delta;i)=\frac{\exp(\delta_1 i+ \delta_2 i^2)}{\sum_{i=0}^p \exp(\delta_1 i+ \delta_2 i^2)}$.
Suppose now $p = 3$, the model will look like this:
$$y_t = \beta_0\Big(1 * x_t + \frac{\exp(\delta_1 1+ \delta_2 1^2)}{\exp(\delta_1 1+ \delta_2 1^2)+1}x_{t-1}+ \frac{\exp(\delta_1 2+ \delta_2 2^2)}{\exp(\delta_1 2+\delta_2 2^2)+\exp(\delta_1 1+ \delta_2 1^2)+1} x_{t-2}\Big)$$
How do I use Non-Linear Least Squares to find the estimates of the parameters $\beta_0$,$\delta_1$ and $\delta_2?$
I have read something about the Gaussian-Newton Method to estimate these parameters. This iteration method is used to minimize the sum of squared residuals:
$$\operatorname{argmin}\sum_{t=1}^T \left(y_t-\left(\beta_0\sum_{i=1}^p w\left(\delta;i\right)x_{t-i}\right)\right)^2$$
Besides, on this Wikipedia page: https://en.wikipedia.org/wiki/Non-linear_least_squares they suggest constructing a Jacobian matrix $J_{ij}$ such that the problem becomes linear and the estimator becomes:$(J^TJ)\Delta\beta=J^T\Delta y$ consequently $\Delta\beta =(J^TJ)^{-1}J^T\Delta y.$
However I feel like this is not possible for my problem, therefore I should use the Gaussian method. Can anyone please give me extra information on this topic?
Picture for methodology:
This should be a comment, but I just need to see that the formula looks right. The formula for the weights is causing you confusion. Try this (I've changed the summation index) $$w(\delta;i)=\frac{\exp(\delta_1 i+ \delta_2 i^2)}{\sum_{j=0}^p \exp(\delta_1 j+ \delta_2 j^2)}$$ Then, for $p=2$ $$y_t=\beta_0\frac{1}{1+\exp(\delta_1+\delta_2)+\exp(2\delta_1+4\delta_2)}(x_t+\exp(\delta_1+\delta_2)x_{t-1}+\exp(2\delta_1+4\delta_2)x_{t-2})$$