The net difference between a smooth action of a topological group $G$ and a representation is the linearity in the space on which they act?
For $X$, say, a differentiable manifold we define the continuous maps:
$Action:G\to \operatorname{Diffeo}(X)$
$Repr:G\to \operatorname{Aut}(X)$ $\hspace{0.07cm}$?
Edit: linearity instead if invertibility, $Diffeo$ instead on $C^\infty$.
An action is by definition always by invertible operators. The difference is that a representation requires $X$ be a vector space and for $G$ to act on $V$ by linear operators.