I am interested in infinite groups $G$ that act on $\mathbb{R}^n$, with some informal properties:
For each $g \in G$, the map $x \mapsto gx$ is smooth and differentiable with respect to $x$.
For each $x \in \mathbb{R}^n$, the map $g \mapsto gx$ is smooth and differentiable with respect to $g$.
Is there a standard concept that is a reasonable match to this or way to formalize this?
Should I perhaps be looking at Lie groups? I confess I've tried learning about them and the examples look like a match but even understanding the definition of a Lie group requires machinery that I am not familiar with.
I realize this is specified in a somewhat vague and imprecise way. I have the sense that my situation falls within some well-studied mathematical concept, but I'm not entirely sure what.
The right general concept seems to be the one of a Lie group, but in many cases the concept of a matrix group is sufficient. To study matrix groups, you observe that the invertible $n\times n$-matrices form an open subset of $\mathbb R^{n^2}$ so the standard concepts of analysis can be applied to them. Matrix groups are then defined as subgroups in there which are closed in the sense of topology. (This is a condition that is very easy to verify in most cases.) A fundamental theorem then says that such a matrix group is a smooth submanifold of $\mathbb R^{n^2}$. Hence in contrast to Lie groups, it suffices to understand submanifolds of Euclidean spaces rather than abstract manifolds, which makes things much easier from a technical point of view. There are several books that develop the theory of matrix groups on a relatively elementary level.