Cauchy point,derivation: whe the constrianed optimizitaion is not used

193 Views Asked by At

Sorry for the slightly longer question. Consider the following definition of the Cauchy point

$h_{i}^{C}=\alpha_{i}^{C}h_{i}.$

It can be found minimizing a quadratic form

$\alpha_{i}^{C}=\arg\min\left\{ f(x_{i})+\nabla f(x_{i})h_{i}^{T}+\frac{1}{2}h_{i}^{T}B_{i}h_{i}\right\} ,\qquad\mbox{subject to }\left\Vert \alpha_{i}^{C}h_{i}\right\Vert _{2}\leq\Delta_{i}$

over the line segment

$h_{i}^{C}=\arg\min\left\{ f(x_{i})+\nabla f(x_{i})h_{i}^{T}\right\} ,\qquad\mbox{subject to }\left\Vert h_{i}\right\Vert _{2}\leq\Delta_{i}$

The condition for the local minimum can be found easily using the Lagrangian

$h_{i}=-\Delta_{i}\frac{\nabla f(x_{i})}{\left\Vert \nabla f(x_{i})\right\Vert _{2}}.$

I am afraid, I do not fully understand, why the Cauchy point is computed from the global minimizer

$\mu_{i}(\alpha h_{i})=f(x_{i})+\alpha\nabla f(x_{i})h_{i}^{T}+\frac{\alpha^{2}}{2}h_{i}^{T}B_{i}h_{i},$

where

$\mu_{i}(\alpha^{\prime})=\nabla f(x_{i})h_{i}^{T}+\alpha h_{i}^{T}B_{i}h_{i}=0$,

and

$\alpha=-\frac{\nabla f(x_{i})h_{i}^{T}}{h_{i}^{T}B_{i}h_{i}}$.

I think that the correct method should use the Lagrangian (it is the constrained optimization)

$\mu_{i}(\alpha^{C}h_{i})=f(x_{i})+\alpha\nabla f(x_{i})h_{i}^{T}+\frac{\left(\alpha\right)^{2}}{2}h_{i}^{T}B_{i}h_{i}+\lambda\left(\alpha\left\Vert h_{i}\right\Vert _{2}-\Delta_{i}\right),$

where

$\mu_{i}(\alpha^{\prime})=\nabla f(x_{i})h_{i}^{T}+\alpha h_{i}^{T}B_{i}h_{i}+\lambda\left\Vert h_{i}\right\Vert _{2}=0$

and

$\alpha=-\frac{\lambda\left\Vert h_{i}\right\Vert _{2}+\nabla f(x_{i})h_{i}^{T}}{h_{i}^{T}B_{i}h_{i}}$

From the back substitution

$\lambda=\frac{-\nabla f(x_{i})h_{i}^{T}\left\Vert h_{i}\right\Vert _{2}-\Delta_{i}h_{i}^{T}B_{i}h_{i}}{\left\Vert h_{i}^{t}\right\Vert _{2}^{2}}$

and finally

$\alpha=\frac{\Delta_{i}}{\left\Vert h_{i}\right\Vert _{2}}.$

However, only the first method is mentioned in papers... Why? In my opinion, it is the constrained optimizing problem. Thanks for your help.