We define the theory ZFC-Infinity+PA as follows. We start with the axioms of ZFC-Infinity. Next we assert that there is a model of arithmetic $(\mathbb N, 0, S, +, \times)$. Next, for every axiom of (first-order) PA, we assert that $(\mathbb N, 0, S, +, \times)$ obeys it. Each axiom of PA corresponds to a separate axiom of ZFC-Infinity+PA.
Note in particular that the theory does not know that $(\mathbb N, 0, S, +, \times)$ is a model of PA, since proving that would require invoking infinitely many axioms.
We do know that $\mathbb N$ is dedekind-infinite though, since the axioms of PA imply $S$ is a bijection between $\mathbb N$ and $\mathbb N - \{0\}$.
So, can this theory prove that PA is consistient?
Your theory proves Infinity and so in particular proves all consequences of ZFC (including Con(PA)). This is pretty much immediate: the set $\mathbb{N}$ for your model must be infinite (for instance, because there is a function $\mathbb{N}\to\mathbb{N}$ which is injective but not surjective; this follows from finitely many axioms of PA).
Alternatively and more directly, let $s:\mathbb{N}\to\mathbb{N}$ be the successor map, and let $A$ be the set of all $n\in\mathbb{N}$ which are in every set which contains $0$ and is closed under $s$. Then you can prove as usual that $A$ satisfies the second-order Peano axioms (this is pretty much immediate from its definition), and so gives a model of PA.