I understand that one way of defining a mathematical object such as a group is to take an object we already know to exist, for example the integers, and take away some properties from them. This is not the only way, and perhaps not the best way, but it is a way.
Are we allowed in ZFC to make definitions in such a way? We may simply define whatever we want so long as there exists some object for which this definition holds ?
We can define the (possibly proper) class of all objects that satisfy the properties of interest. For example, we can define the class $\rm SemiGrp$ of all pairs $\langle X,+\rangle$ such that $+:X\times X\to X$ and $x+(y+z)=(x+y)+z$ for all $x,y,z\in X$. All this does is give a name to the property, and most properties defined this way are proper classes. There does not even need to exist an object satisfying the property - for example the class of all inaccessible cardinals may be a proper class, or it may be empty - but one can define the class itself with no issues.
There are properties that we cannot even define in this sense, for example the set of all Godel numbers of true formulas, because it would lead to a contradiction to have a totally faithful interpretation of the truth predicate (see Undefinability of Truth); the issue arises in trying to model ZFC within itself, because this leads to a proper class which you can't then quantify over in the way one would like.