The "Right" Concrete Category

Question

One thinks of cardinality as being a property of groups, topological spaces and the objects of many other common categories. This is typically expressed through affixing to the category $\mathcal{C}$ of groups, topological spaces, etc. a "standard" functor that is injective on morphisms, $F:\mathcal{C}\to\textbf{Set}$, e.g. when groups, topological spaces, etc. are defined as sets, and the morphisms between them are defined as functions, $F$ just sends objects back to the sets they were defined as, and morphisms back to the functions they were defined as. Whatever functor is chosen, the view that cardinality is a property of objects in $\mathcal{C}$ is at least vindicated with respect to $F$, as isomorphic objects in $\mathcal{C}$ will be mapped to isomorphic objects in $\textbf{Set}$. However, in general, $\mathcal{C}$ may have many such functors to $\textbf{Set}$ which do not agree on the cardinalities of objects. As such I was wondering if there is formal definition of the "right" way to view a particular category as a concrete category, presumably using a universal property.