Let $G$ be a Lie group that is a closed subgroup of the isometry group of a manifold $M$. Identify each $X$ in the Lie algebra $\mathfrak g$ of $G$ with the vector field on $M$ that is generated by the one-parameter group of diffeomorphisms $\phi_t(y) = \exp(tX)y$. Then Arthur Besse’s book “Einstein Manifolds” warns on p.182 that $$[X,Y]_{\mathfrak g} = -[X,Y],$$ where $[\cdot,\cdot]$ is the bracket of vector fields on $M$ and $[\cdot,\cdot]_{\mathfrak g}$ is the bracket on $\mathfrak g$.
Attempt: By the definition of the commutator on $M$, $[X,Y]f$, where $f$ is a function on $M$, is $(XY - YX)f$. The first term, $XY(f)(p)$, equals
$$\left.\frac{d}{dt}\right\vert_{t=0}(\exp tX)^*\left.\frac{d}{ds}\right\vert_{s=0}(\exp sY)^* f(p) = \left.\frac{d}{dt}\right\vert_{t=0}\left.\frac{d}{ds}\right\vert_{s=0}(\exp tX)^*(\exp sY)^*f(p) = \left.\frac{d}{dt}\right\vert_{t=0}\left.\frac{d}{ds}\right\vert_{s=0}(\exp tX)^*f((\exp sY)(p)) = \left.\frac{d}{dt}\right\vert_{t=0}\left.\frac{d}{ds}\right\vert_{s=0}f\big((\exp sY)((\exp tX)(p))\big) = \left.\frac{d}{dt}\right\vert_{t=0}\left.\frac{d}{ds}\right\vert_{s=0} f\left(\exp \left(sY + tX +\frac{st}{2}[Y,X]_{\mathfrak g} + \dots\right)p\right),$$
by the BCH formula. So far it looks promising that the result of these two derivatives combined with the $-YXf$ term should yield $[X,Y]f = [Y,X]_\mathfrak g f$. Unfortunately, I’m unsure how to apply the chain rule correctly when taking these two derivatives. In my attempt, I got $XY(f)(p) = f’’(p)XpYp +f’(p)\left(\frac 12 [Y,X]_\mathfrak g + YXp\right).$
Note: Besse gives two references for this warning, but unfortunately, I couldn’t find them. The first is N.R. Wallach’s “Compact homogeneous Riemannian manifolds with strictly positive curvature” and the second is Kobayashi and Nomizu’s “Foundations of differentiable Geometry Vol. II” p.469 (this reference doesn’t even have this many pages).
I do not have the book to consult, but typically this sign problem crops up when you use right-invariant vector fields rather than left-invariant. And that's exactly the case you've described. If $X$ is a right-invariant vector field, then its flow is given by $\varphi_t(p)=L_{\exp tX}(p)$.
EDIT: As I suggested in the comments, consider the concrete calculation of the Lie derivatives in $GL(n)$. Compare $$\left.\frac d{dt}\right|_{t=0}(e^{-tA}Be^{tA}x) \qquad\text{and}\qquad\left. \frac d{dt}\right|_{t=0}(xe^{tA}Be^{-tA}).$$