Spectral coordinates on Hermitian matrices

Question

Both in Venezianni and Pereira, A note on the volume form of normal matrix space, it is claimed that the decomposition of a normal matrix $M=U\Lambda U^\dagger$ with $\Lambda=\operatorname{diag}(\lambda_1,\dots,\lambda_N)$ and $U\in U(N)$ is unique up to multiplication by phases of the rows of $U$. However, there is also an ambiguity in the ordering of the eigenvalues and the rows of $U$, right? Why can we forget this ambiguity to create a chart on the space of normal matrices based on $\Lambda$ and $U$ that covers an open set of full measure?