I was told that the power set of an ordinal is not an ordinal in general.
However, if $\alpha$ is ordinal, then
$$\alpha+1 = \alpha \cup \{\alpha\}=\{\beta: \beta \in \alpha\} \cup \{\alpha\}= \{\beta: \beta \subsetneq \alpha\} \cup \{\alpha\}: \{\beta: \beta \subseteq \alpha\} = \mathcal{P}(\alpha)$$
Where do I go wrong? Is it wrong because the $\beta's$ are ordinals and not every subset of an ordinal is an ordinal?
It's not the case that $\alpha \in \beta \Leftrightarrow \alpha \subsetneq \beta$. For example, $\{ 0, 2 \} \subsetneq 3$ but $\{ 0, 2 \} \not\in 3$.