I have a really stupid question, which I can't figure out at the moment.
I don't see why the following is correct when you check the dimensions:
$\tau = I \alpha \;\;\rightarrow \mathrm{Nm= kg \;m^2 \frac{rad}{s^2 } } \leftrightarrow \mathrm{Nm=Nm \; rad}$ (where the last equality doesn't seem correct) .
What do I do wrong here?
Here you can find the formula, with explanation
The radian is what in the specification of metric units is called a supplemental unit, and it doesn't behave like a regular unit.
Consider the formula for the length $s$ of an arc of circle of radius $r$ subtended by the (central) angle $\theta$ in radians:
$s = r \theta$.
Here we run into the same issue that you do in your dimensional analysis: On the left hand side we have (for example) $\textrm{m} \cdot \textrm{rad} = \textrm{m}$. What's going on here is that the radian is defined precisely so that the above equation will hold true when $\theta$ is measured in radians, so we can safely leave it out of our dimensional analysis. That applies just as well if we differentiate an angular quantity, or differentiate twice and produce an angular acceleration $\alpha$ like in your formula.