Let $G$ be a Lie group (given by a matrix). Let $\frak g$ be its Lie algebra.
I would like to know if the following is true.
"Let $X$ be a matrix in $\frak g$. Then $\gamma(t)=\exp(tX)$ is a curve in $G$."
Here $\exp(Y)=\sum_{n=0}^{\infty}\frac{Y^n}{n!}$ for a matrix $Y$.
Consider the ODE on $G$: $$ \gamma^{-1} \dot \gamma = X , \qquad \gamma(0) = I .$$ Then by definition, this is a well defined equation on the Lie group. This is because $X$ is an element of the tangent space of the identity, and $\dot \gamma$ is an element of the tangent space at $\gamma$, and pre-multiplication by $\gamma^{-1}$ maps the tangent space at $\gamma$ to the tangent space at $I$.
So if the Lie group happens to be a subgroup of the $n\times n$ matrices, it makes no difference - the solution to this ODE, if it exists, stays inside the group.
But the solution to this ODE is given by $\gamma(t) = \exp(tX)$.
(You can use post-multiplication as well $\dot \gamma \gamma^{-1} = X$.)
Another way to see it. For any $X \in \frak g$, and $\epsilon$ sufficiently small, there is a function $h(\epsilon)$ so that $I + \epsilon X + h(\epsilon) \in G$, where $h(\epsilon) = o(\epsilon)$ as $\epsilon\to 0$.
Now consider $\left(I+\dfrac tn X + h\left(\dfrac tn\right)\right)^n$. This is a product of elements of $G$, and hence is in $G$. But this converges to $\exp(tX)$ as $n \to \infty$.