In ZFC minus infinity (let us call this system $T$), one can still define ordinals, and then define integers as ordinals all of whose members are zero or successor ordinals. Combining the power set axiom with the subset axiom scheme, one can also define the cartesian product of two sets (using the Kuratowski pair $(a,b)=\lbrace \lbrace a \rbrace, \lbrace a,b \rbrace \rbrace$, $A\times B$ is a subset of ${\cal P}({\cal P}(A\cup B))$).
Consider the statement $\phi$ to the effect that if $\alpha,\beta$ are any two integers (in the above sense) then there is another integer $\gamma$ such that there is a bijection between $(\lbrace 0 \rbrace \times \alpha) \cup (\lbrace 1 \rbrace \times \beta)$ and $\gamma$.
Then $\phi$ is true and can be proven (outside $T$) by induction. It follows from the argument in this answer by Asaf Karagila that $\phi$ is in fact already provable inside $T$.
But notice that Asaf's argument is rather advanced (it uses for example Gödel's "completeness theorem" that every consistent theory has a model) and indirect : it shows that there is an elementary proof inside $T$ without explicitly giving it.
My question is, describe explicitly a completely elementary proof of $\phi$ from $T$.
Can't you just use replacement? Let $S$ be the well order consisting of $\alpha$ followed by $\beta$, as in the question.
Proof sketch: Consider the collection $C$ of all ordinals $x$ such that there is a (necessarily unique) order isomorphism $f_x$ between $x$ and a (necessarily unique) initial segment of $S$. By replacement $C$ is a set. Then $\bigcup C$ is an ordinal, and $\bigcup_{x \in C} f_x$ is a bijection from $L$ to $\bigcup C$.
I don't see any use of the axiom of infinity there.