I'm not sure if an old thread appears in the list if I reused and commented it, but to me it doesn't. Therefore I open a new one and just link my question: Difference between Newton's method and Gauss-Newton method Thx
update: In virtue of the desired Q&A format this i my question with respect to both methods:
I'm wondering why one wants to ignore the Hessian in the nonlinear least square quasi newton method? Whats the advantage? Does it ensure better convergence for some reason? I don't see why dropping this term is worth considering? It's not a huge issue to include the hessian...So there must be some deeper reason that for example it seemed to be more robust in iterative methods compared to including the hessian, but convergence is slower...(newton is 2. order converging,or?) compare the update process: $ \Delta b = -\left( J^T J + H r(b) \right )^{-1} J^T r(b) $ with and without the hessian $H$...