I have this issue with the convergence of the quasi newton method. I have a convex objective function which I need to minimize wrt some parameters. I generated some synthetic data using a defined params. Now I want to optimize using quasi Newton method to retrieve the params. However, it tends to get stuck
I am using armijo rule for finding the step size. With every iteration, I approximate the hessian using quasi Newton method. However, at times, when I run the code, it tends to get stuck at a point which is not optimum. I mean armijo rule requires that there is sufficient decrease in the objective for step size alpha. However, at times, I reach certain point using the quasi newton based descent direction and I get stuck at that point because even with very small value of alpha to the range of ^-100, there is no decrease in the objective function.
I guess it is because of the approximate Newton direction. Is it so? Can anyone provide some suggestion about the convergence of quasi newton method?