I have a vector valued function $F: \mathbf{R}^{n} \rightarrow \mathbf{R}^{m}$, which consists of quadratic taylor approximations. So one could say that function F consists of stacked approximations like: $F_i = f(\pmb a) + \nabla f(\pmb a)^T (\pmb x - \pmb a) + 0.5 (\pmb x - \pmb a)^T \nabla^2 f( \pmb a ) (\pmb x - \pmb a) \quad \forall i=1,...,m$ $F = [F_1, F_2, ..., F_m]^T$
currently I am solving the sum of squares problem of this with Gauss-Newton:
$\min \left\Vert F \right\Vert_2^2$
but Gauss-Newton uses only an approximation of the Hessian matrix: $H \approx 2( \nabla f(\pmb a) )^T ( \nabla f(\pmb a) )$.
Since I can explicitly express the Hessian, and the approximation for the Hessian is quite bad, I would like to use exact Hessian when possible.
Do you know any method to solve this problem?
Thank you!
EDIT: What about solving the sum-of-squares problem with Newon's method (true Hessian) ? I.e:
$g=\sum_{i=1}^{m} f_i^2 = \left\Vert F \right\Vert_2^2$ and then:
$x_{k+1} = x_k - ( \nabla^2 g )^{-1} \nabla g$
is this a good idea? Why does one normally use Gauss-Newton over Newton's method for this? Only for computational complexity reduction?