Let $\alpha$ be an ordinal then $\alpha=\cup\alpha$ or $\alpha=s(\cup\alpha)$.
Attempt:
Since $\alpha$ is transitive $\alpha=\cup\alpha$ (here we are done) or $\alpha\supsetneq\cup\alpha$.
If $\alpha\supsetneq\cup\alpha$, since $\cup\alpha$ is an ordinal we have $\cup\alpha\in\alpha$. Regularity axiom implies $s(\cup\alpha)=\alpha$ (here we are done) or $s(\cup\alpha)\in\alpha$.
But I can't see any contradiction assuming $(\cup\alpha\in\alpha) \land (s(\cup\alpha)\in\alpha)$.
To get a contradiction from $s(\bigcup\alpha)\in\alpha$, note that, by definition of $s$, we also have $\bigcup\alpha\in s(\bigcup\alpha)$. So $\bigcup\alpha$ is a member of a member of $\alpha$. By definition of $\bigcup$, we have $\bigcup\alpha\in\bigcup\alpha$, contradicting the axiom of regularity.