Wiki says that the group $U(n)$ is a semi-direct product of $SU(n)$ and $U(1)$.
Each element $g$ of a semi-direct product $G=HK$, should be uniquely represented as $g=hk$.
$SU(n)$ and $U(1)$ have common elements, $\operatorname{e}^{i \frac{m}{n}2\pi} \mathbf{1}$. This allows for multiple expressions for elements.
What am I doing wrong?
According to the same wikipedia article:
This copy of $\mathrm{U}(1)$ and $\mathrm{SU}(n)$ intersect only at the identity.