Yesterday I asked this question in MathOverflow but did not receive an answer yet. I want to try my chance here too, since I am in kind of a hurry. Answers will be much appreciated.
I intend to propose as a project the proof of the statement below, but I want to make sure that it is not already proved somewhere else before.
Let $G$ be a Lie group, and $f$ a real-valued function on $G$. The expression $f \in \mathcal{C}^k(G)$ makes sense, and this would be the case even if $G$ were merely a smooth manifold. On the other hand, the Lie group structure on $G$ enables one to speak of directional derivatives of $f$. Indeed, the Lie algebra $\mathfrak{g}$ of $G$ is canonically isomorphic to the space of left-invariant derivations of $\mathcal{C}^\infty(G)$; under this isomorphism, each $X \in \mathfrak{g}$ is associated with the (left) Lie derivative operator $\mathcal{L}_X$ given by $$(\mathcal{L}_X f)(y) := \left.\frac{\mathrm{d}}{\mathrm{d} t}\right|_{t=0} f(y e^{tX})$$ where $f \in \mathcal{C}^\infty(G)$ and $y \in G$. It is natural to call $\mathcal{L}_X f$ as the (left) directional derivative of $f$ along $X$. Taking the above equality as a definition, one may expect as in elementary analysis that being in $\mathcal{C}^k$ is equivalent to having continuous directional derivatives of order $k$. This is what our statement says:
Statement. Let $G$ be a Lie group, and $f$ a real-valued function on $G$. For each $k \in \mathbb{N}$, $f \in \mathcal{C}^k(G)$ if and only if $(\mathcal{L}_{X_1} \cdots \mathcal{L}_{X_k})f$ exists and is continuous for all $X_1,\ldots,X_k \in \mathfrak{g}$.
I searched quite a while for this statement in the literature but could not find anything. (The proof is not so trivial as you might think at a first glance. Please have a look at this question and p. 15 of this essay also.) Have you ever seen it somewhere? If so, could you please give a reference?
Let us consider what happens on $\mathbb{R}^n$. We denote by $X_1,\dots,X_n$ the standard vector fields. Then it is well known that $f$ is $C^k$ iff for any $i_1,\dots,i_k$, the directional derivatives $X_{i_1}.(X_{i_2}(\dots.(X_{i_k}.f)\dots)$ exist and are continuous.
Now take any other vector fields $Y_1,\dots,Y_n$ giving a basis of $T_x \mathbb{R}^n$ for $x$ in a neighborhood of $x_0$. Then you can write $Y_i = \lambda_i^j X_j$, with smooth $\lambda_i^j$. This shows that if $f$ is $C^k$ in a neighborhood of $x_0$ then the directional derivatives $Y_{i_1}.(Y_{i_2}(\dots.(Y_{i_k}.f)\dots)$ exist and are continuous. Reversing the argument, this shows that in order to show that a map is $C^k$, you can compute directional derivatives for any vector fields giving a basis of the tangent space.
Now, for your Lie group, $L_X f$ is just the directional derivative of $f$ in the direction of the left-invariant vector field associated to $X$. I think this proves what you want. Or am I missing something ?