Levenberg-Marquart with Hessian and gradient

1.1k Views Asked by At

I am minimizing a sum-of-squared-differences function using the Levenberg-Marquardt method. The off-the-shelf numerical implementations I have have looked through (MATLAB, Numerical Recipes in C (15.5), this implementation) all ask for a definition of the Jacobian. However, the paper that I am implementing gives only derivations for the gradient vector and Hessian matrix (pg. 4).

While the paper might give more clarity to my question, the short of it is that the function is an error function of the form:

$$\frac{1}{n}\sum_{i=1}^{n}[I(x,y,t) - I(x', y', t+1)]^2 = \frac{1}{n}\sum_{i=1}^{n}e_i^2 $$

where $I(x,y,t)$ is known and $I(x',y',t+1)$ is a function of some vector $\mathbf{f}$.

The objective is thus:

$$argmin_{\mathbf{f}} \left( \frac{1}{n}\sum_{i=1}^{n}e(\mathbf{f})_i^2 \right)$$

The paper says it to use Levenberg-Marquardt and provides:

...the approximate Hessian H and the gradient vector G can be computed,

$$\mathbf{h_{m,n}} = \sum_{k} \frac{\partial e_k}{\partial f_i}\frac{\partial e_k}{\partial f_j}, \mathbf{g_i} = \sum_{k}\frac{\partial e_k}{\partial f_i} $$

All of this brings me to my question:

Why might I need this Hessian approximation and gradient for Levenberg-Marquardt? In general doesn't L-M just need the Jacobian which is the essentially this gradient without summing each column $i$ (ie. $\mathbf{J_{k,i}} = \frac{\partial e_k}{\partial f_i}$)?