I am doing gradient descent with respect to se loss function L. I have found that the partial derivative wrt some of the parameters is zero whilst the covariant derivative wrt the same parameters is nonzero.
Is this possible?
I can imagine the connection components (christoffel symbols) introducing additional terms related to the curvature of the manifold but I am struggling to interpret what this "looks like". If the partial derivative wrt parameter vector X is zero, that means X is orthogonal to level sets. If that's true everywhere then the manifold should be completely flat? Or am I just not visualising how curvature could enter into the picture.
Appreciate any help as I'm pretty confused!