Why is it that a finite integral domain $R$ can only have cardinality $p^n$ for some prime $p$? I'm familiar with the elementary result that a finite integral domain must be a field, and after one observes that a finite field must have prime characteristic for some prime $p$, then we can easily equip the field with a vector space structure over $\mathbb{Z} / p \mathbb{Z}$, and so it's a matter of linear algebra that this integral domain must have cardinality $p^n$, where $n$ is its dimension as a $\mathbb{Z} / p \mathbb{Z}$-vector space.
All of this, however, relies on the machinery of fields. I of course have no concerns about the "legality" of this. That finite integral domains are fields is a matter of the most elementary combinatorics. But what interests me is that though it's somewhat clear that a finite field must have particular cardinality, it's much less obvious that a finite integral domain must have these constraints. Is there an intuitive reason why an integral domain should have these properties without recourse to their field structure?