I am reading this paper dealing with optimization on Lie groups and in the introduction (second full paragraph of the second page) the authors write:
''However, a specialization in Lie group will still be helpful, because the additional group structure (joined efforts with NAG) improves the optimization; for instance, a well known reduction is to, under symmetry, pull the velocity at any location on the Lie group back the tangent space at the identity (known as the Lie algebra).''
With my limited background of differential geometry I am trying to understand this statement formally and how I can use it to design optimization algorithms (e.g. Riemannian gradient descent) when my cost function is defined on a Lie group.
Let me explain my attempt to understand how this works. Let $\mathcal{G}$ be a Lie group and $f : \mathcal{G} \to \mathbb{R}$ a smooth function. The Lie group $\mathcal{G}$ can be made into a Riemannian manifold by defining $\forall g \in \mathcal{G}$ the Riemannian metric $$ \langle X, Y \rangle_g = {\rm trace}(XY^t), \qquad \forall X,Y \in T_{g} \mathcal{G}. $$ I would like to obtain the Riemannian gradient of $f$ at $g \in \mathcal{G}$ with respect to the metric defined above. To do so I consider the pullback of $f$ by $L_g$ (left translation by $g$) $$ L_g^{\ast}f = f \circ L_g, $$ and compute the gradient $V = {\rm grad} (L_g^{\ast} f)(e)$ at the identity $e \in \mathcal{G}$ which is an element of the tangent space $T_e \mathcal{G}$.
If I multiply $V$ on the left by $g$ then I get $g V$ which is an element of the tangent space $T_g \mathcal{G}$ and I would like to think that $gV$ is the gradient of $f$ at $g$? Why would such a claim be true, something to do with left-invariant vector fields?
Clearly I do not fully understand what is going on here and I am lacking proficiency in the language of differential geometry. Any comments which will help clarify my understanding would be greatly appreciated.