Say I start with the cartesian coordinate system, $xyz$. I introduce a new coordinate system, $x'y'z'$, where, viewed from the first coordinate system, $x'$ and $y'$ point along the $(-2,1,0)$ and $(1,2,-1)$ directions, respectively.
How does one figure out the angle of rotation?
If $R$ is the matrix of a 3-d rotation, then $\operatorname{tr}R = 1+2\cos\theta$, where $\theta$ is the rotation angle about an axis the corresponds to the eigenspace of $1$. You already have two of the columns of this matrix: they are the direction vectors of the $x'$ and $y'$ axes, normalized to have unit length. The unit vector giving the direction of the $z'$-axis is the third column of $R$, which you can find by taking the cross product of the other two columns.