ZFC doesn't have finitely many axioms because it has some axiom schemas. Let's look at Axiom Schema of Specification:
$$ \forall X \exists Y \forall a \quad (a \in Y \iff a \in X \land P(a)) $$
This specifies a different axiom for each predicate $P$. Since $P$ is essentially a function from $X$ to boolean values, taken literally as $X$ is an arbitrary set, this axiom schema specifies uncontainably many axioms. That is, the class of axioms this axiom schema specifies is not a set.
Still, I think this axiom schema actually specifies countably infinite axioms, because we can construct an $X$ and a $P$ only in finitely long "text" with finitely many "characters". Is this the case?
In the schema that you've written $P$ needs to be a formula of the language of set theory. The language of set theory is finite, and formulas are finitely long, so there are only countably many $P$ available to consider.
In other words, not every function from $X$ to $\{T, F\}$ is eligible to be $P$ in the schema.