In my book the following definition is given:
A category $C$ is called small if both the collection of objects and arrows are sets. Otherwise the category is called large.
A set is delfined to be a collection of distinct objects. Now I am a little confused since the category of all finite sets is said to be small, which is ok for me since the objects and functions can be considered as sets. But why isn't the category of groups a small category? All groups may be considered as distinct objects and hence a set, same argument goes for group homomorphisms. Can someone explain why I am so confused here?
In set theory (ZFC), you cannot construct the set of all sets. That's why the need to distinguish between big and small categories: in the big ones you cannot use some machinery, otherwise we would run into some problems like Russel's Paradox.
Since the underlying set of a group is a set, you cannot construct the set of all groups as well. You can though construct the set of all finite groups (say, that are subsets of $\mathbb{N}$).