Regression: Use derivative information for better estimate of parameters of model

Question

If additional information of slopes are available in addition to observations; Can this information be used to improve fitting? Following could be considered as an example:

Suppose I have $\mathbf{x}$ and corresponding $\mathbf{y}$ observations; and I need fit and estimate the parameters of the model of form $y = a \mathrm{sin}(bx)+c$; where a, b, c are parameters of model.

Now, somehow I also know derivative values very precisely at $\mathbf{x'}$; where $\mathbf{x'}$ may or may not be equal to $\mathbf{x}$.

Can I improve my parameter estimates using derivative information? (My intution says YES). If yes how?

Answer 1

This is a very interesting post about regression where you also have accurate values for the derivatives at some points.

After JJacquelin's answer, let me suppose that you have $n$ data points$(x_i,y_i)$ and $m$ data points$(x_i,y'_i)$. So, let us consider the models $$y=a+b\sin(\omega x)+ c\cos(\omega x)$$ $$y'=b\omega\cos(\omega x)- c\omega\sin(\omega x)$$ and the sums of squared errors $$S_1=\sum_{i=1}^n\left(a+b\sin(\omega x_i)+ c\cos(\omega x_i)-y_i\right)^2$$ $$S_2=\sum_{j=1}^m\left(b\omega\cos(\omega x_j)- c\omega\cos(\omega x_j)-y'_j\right)^2$$ Then, what you can do is to minimize with respect to the $(a,b,c,\omega)$ parameters the global function $$\Phi=S_1+k S_2$$ where $k$ is a weighting factor you assign to the data (it will say for example that the derivatives are ten times more accurate that the function values).

Since JJacquelin's simple procedure already gives more than reasonable estimates for the parameters, you could start from these using an optimizer or even use a Newton-Raphson method to get final values.

Answer 2

First note : The model $y=a\sin(bx)+c$ can be used in case of the example represented on your graph because it is a particular case. Generally, to model a sinusoidal function of this kind, not only three but four parameters are necessary. The model should be : $$y=a+\rho\sin(\omega x+\varphi)\qquad\text{or :}$$ $$y=a+b\sin(\omega x)+c\cos(\omega x)$$

Second note : Using the derivative is possible but not the best option. Due to the scatter of the data, the numerical calculus of the derivatives is often even more scattered. Generally it is better to use antiderivatives (integrals) which, on the contrary, tends to smooth the scatter.

A straightforward procedure (no initial guess, no iteration) is shown pp. 35-36, with a numerical example in the paper: https://fr.scribd.com/doc/14674814/Regressions-et-equations-integrales

In this paper, the procedures are preferably based on anti-derivatives instead of derivatives in benefit of the robustness of calculus. But similar procedures involving derivatives are possible as well, as it is noted in introduction.

From this paper, an even more simplified method can be extracted :

Of course, it's less accurate than the complete procedure, but sufficient to have a first approximate. This rough result could be used, instead of initial guess, for further non-linear regression methods.

With the data obtained by numerical scanning of the pjk's graph, the result is (blue curve) :

$\omega=3.053 \quad ; \quad a=1.7225 \quad ; \quad b=1.4727 \quad ; \quad c=-0.1078$

The result of the above simplified procedure is to compare with a more advanced calculus (black curve) :

$\omega=2.997 \quad ; \quad a=1.7184 \quad ; \quad b=1.4656 \quad ; \quad c=-0.0274$