The problem I'd like to ask is the following : let $M_1$ and $M_2$ two rigid bodies with a quadratic constraint function $f$ attached to its grid points. $M_2$ is always kept static while $M_1$ can be rotated.
I would like to determine the optimal rotation using quaternions, that is $$\min_{q\in\mathbb{S}^3}f(x(q), r)$$ where $x(q)\in \mathbb{R}^3$ is the rotating grid point in $M_1$ and $r\in\mathbb{R}^3$ is the reference grid point in $M_2$. Now I only added constraint between two points.
I managed to derive (if I'm correct) that the gradient of $f$ is $$\nabla f= \frac{\partial f}{\partial \rho}\cdot\frac{\partial \rho}{\partial q}$$ where $q$ is the euclidean distance between the two points and the derivative of the distance depends only on $\frac{\partial x(q)}{\partial q}$, that is the Jacobian of $x(q)$. Therefore the quaternion dependency only comes is through the dependency of the euclidean distance norm.
My question is whether I can apply a regular non-linear CG optimizer to find the optimal unit quaternion rotation or not. I also managed to figure out that when the optimizer would compute a step $s$ that should be projected back onto $\mathbb{S}^3$ using the canonical $exp$ function. On the other hand I don't know how to initialize the algorithm as at each step the search direction is in $T_{q}(\mathbb{S}^3)$.
I searached through the corresponding literature but I did not find any hint on applying the NCG to find optimal rotation.
Thanks for every hint !