For natural examples of locally small categories like the category $\mathbf{Grp}$ of groups, the isomorphism classes themselves are normally not sets. In a set theory like ZFC, even the collection of a finite number of such isomorphism classes would not be a set, because the members of a set must be sets themselves.
But suppose we had a set theory with ur-elements and classes, which would allow us to form such collections, as long as they are well enough defined. For some categories like the category $\mathbf{FinSet}$ of finite sets, the collection of isomorphism classes will form a nice countable set. I'm not sure what happens for the category of (hereditarily) countable sets. (Maybe the corresponding collection will still be countable?)
For a category like $\mathbf{Grp}$, the collection of isomorphism classes will probably not be countable, but instead will probably be quite "huge" relative to the available set universe. I'm not sure whether it will "always" be a set, but still this collection seems to be much smaller to me than any single isomorphism class of this category. But how can we distinguish a natural category like $\mathbf{Grp}$ where the collection of isomorphism classes is still "reasonably small" from unnatural categories where the notion of isomorphism is too restricted, such that the collection of isomorphism classes cannot be a set, even for the most powerful set theories?
One way to circumvent this kind of issue is to work with Grothendieck's universes introduced in SGA4 (as an appendix by Bourbaki if I remember correctly).
For any universe $\mathbb U$, define $\mathbb U$-categories as categories where the hom-sets are elements of $\mathbb U$. So you have the categories $\mathbb U{-}\mathsf{Set}$, $\mathbb U{-}\mathsf{Grp}$, etc.
Then the most important axiom of Grothendieck universes is : for any naive set, there exists a universe containing it. Apply it to, say, the collection $\operatorname{Ob}(\mathbb U{-}\mathsf{Grp})$ which is in some universe $\mathbb V$ : the category $\mathbb U{-}\mathsf{Grp}$ is then a small $\mathbb V$-category.
In that framework, there is no unnatural category !