I am currently working on a code that optimises (minimises) a function. In simple terms I have a scalar function $f(x_1,x_2,x_3...)$, and its gradient $\nabla f$ and Hessian $\nabla^2 f$. The optimisation is through a quasi-Newton method (second-order).
The gradient is obviously a vector: $$\vec{g} = \nabla f = \left[\frac{\partial f}{\partial x_1}, \frac{\partial f}{\partial x_2},...\right]$$
Now I am trying to optimise along one component of the gradient only. So let's say there is a vector $\vec{a}$, and you get two components of the gradient, one along $\vec{a}$ and the other perpendicular to it.
$$\vec{g} = \vec{g}_\parallel + \vec{g}_\perp $$
where
$$\vec{g}_\parallel = \frac{\vec{g}\cdot\vec{a}}{\vec{a}\cdot\vec{a}}\vec{a}$$.
I want to optimise $f$ only along the perpendicular component of the gradient vector, or you could say that I want to zero the parallel component of gradient vector. What would be the expression of the Hessian along this direction, and how to derive it from the original Hessian?