I am having trouble verifying that $\alpha \cdot \alpha^\omega =\alpha^\omega$ for any ordinal $\alpha$.
The case is trivial for $\alpha = 0, 1$. So for $\alpha >1$ we get $\alpha^\omega = \bigcup_{n<\omega}\alpha^n$, which I beielve is a limit ordinal.
But I'm not sure how to figure out $\alpha \cdot \bigcup_{n<\omega}\alpha^n$.
According to what you said you only need help figuring out the last one.
The idea is to show that $\alpha\cdot \alpha^\omega = \alpha^{1+\omega}$ and from here, since $1+\omega =\omega $, you get what you want.
The important thing is that multiplication on the left is continuous by definition.
So $\alpha\cdot \alpha^\omega = \displaystyle\bigcup_{\delta< \alpha^\omega }\alpha\cdot \delta$, by definition. But for every $\delta <\alpha^\omega$, there exists $n\in \omega$ such that $\delta\leq \alpha^n$,and so (this you should prove) $\alpha\cdot \delta \leq \alpha\cdot \alpha^n$, and so
$\displaystyle\bigcup_{\delta< \alpha^\omega }\alpha\cdot \delta \leq \displaystyle\bigcup_{n<\omega }\alpha\cdot \alpha^n = \alpha^{1+\omega}$ by definition of $1+\omega$ and ordinal exponentiation (you should prove that $\alpha\cdot \alpha^n = \alpha^{1+n}$ for finite $n$). So $\alpha^{1+\omega}\geq \alpha\cdot\alpha^\omega$.
You can establish the reverse equality by noting that if $\alpha \geq 2$, then for every $n\in \omega$, $\alpha^n<\alpha^\omega$, and then do the same proof as I just did, but in the other direction.