I have a function "fkt." In this example, let it be as simple as $y=a \cdot x+b$.
I have a real dataset with values obeying to the model. After regression of the points to the model, I find the regression parameters $a$ and $b$.
The recomputed values deviate from the dataset in the residuals $R$. The sum of the squared residuals are $rss$.
I have the following problem: when I calculate the Hessian Matrix as \begin{equation} \textrm{H} = \frac{\partial^{2}rss}{\partial \textrm{p}^{2}} = \left[ \begin{array}{cccc} \frac{\partial^{2}rss}{\partial {p_{1}}^{2}} & \frac{\partial^{2}rss}{\partial {p_{1}}^{2}\partial p_{2}} & \cdots & \frac{\partial^{2}rss}{\partial {p_{1}}^{2}\partial p_{np}} \\ \frac{\partial^{2}rss}{\partial p_{2}\partial p_{1}} & \frac{\partial^{2}rss}{\partial {p_{2}}^{2}} & & \frac{\partial^{2}rss}{\partial p_{2}\partial p_{np}} \\ \vdots & & \ddots & \\ \frac{\partial^{2}rss}{\partial p_{np}\partial p_{1}} & \frac{\partial^{2}rss}{\partial p_{np}\partial p_{2}} & & \frac{\partial^{2}rss}{\partial {p_{np}}^{2}} \end{array} \right] ,\end{equation}
I gain values twice as high as they should be.
The idea that they are to high come from the estimate of the Hessian through the Jacobian, which is calculated as the following: $J = \frac{\partial R}{\partial p}$, because H is approximately $J^{T} \cdot J$.
As the model is very simple the standard deviation of the slope and bias might be calculated analytically by hand. The values gained by doing so are the same if I used the values for the Hessian I gained through the Jacobians and are not the same if I would use the real, self calculated Hessian.
When I alter the Formula of the Hessian to:
$\textrm{H} = \frac{\partial^{2}\frac{rss}{2}}{\partial \textrm{p}^{2}}$
everything is fine. But I can't see this devision by 2 anywhere in literature.
Can you please help me? Why is the direct calculated Hessian an other one as the one estimated through the Jacobians, and why is it exactly by the factor of two to high?