I'm reading a book on commutative algebra, where there is an appendix on category theory.
It defines a category $\mathcal{C}$ as having a set of objects $\text{Ob}(\mathcal{C})$ and a set of arrows between objects $\text{Hom}(\mathcal{C})$ satisfying conditions of composition, identity and associativity.
I'm a little troubled because I don't see how we can define the category of sets this way.
Don't we have the classical paradox of "the set of all sets"?
What am I missing?
We do not run into that "classical paradox", known as Russell's paradox, because we have the notion of class. Class is a broader notion of collection than set; all sets are classes, but not all classes are sets. Classes which are not sets are called proper classes. In this framework, what Russell's paradox shows is that the class of all sets is a proper class, because if the class of all sets was a set we would get a contradiction. Now you might be wondering, why can't we form a new Russell paradox about the class of all classes? The answer is that there is no class of all classes, because not all classes are allowed to be the elements of a class. Only a set can be an element of a class, a proper class cannot be an element of a class. And how do you tell which classes are sets and which ones are proper classes, short of deriving a contradiction? The approach mathematicians have settled on (though by no means the only consistent approach) is to look at its "size". Proper classes are all of the same size; they can all be put in one to one correspondence with the class of all sets. And if a class is too small to be put in one-to-one correspondence with the class of all sets, i.e. it has an actual cardinality, then we call it a set. This is all formalized in set theories like NBG and MK.
So to answer your question, it's not a problem that all sets do not form a set. Because the objects in a category just need to form a class, not a set. In fact categories where there is a set of objects rather than a proper class of objects have a name, small categories. And we can even create a category of all small categories, with functors as morphisms. That's legitimate because small categories are sets, so we can form a class out of them. But obviously the category of small categories is not a small category.
So to sum up, the category of sets is a perfectly legitimate category, it's just not a small category.