I'm dealing with rotation about an arbitrary axis and I know the vector of this axis and angle that I want to rotate. Is there a way to calculate angles of this rotation into a rotation about an XYZ axis.
Let's say I have axis with vector (1, 1, 1) and 30° angle, what would be angles for x, y and z rotation using basic matrices for rotation, so I would get the same result.
Thanks
On
The most useful way to represent rotations in 3D is using quaternions. In such representation a rotation of an angle $2\theta$ in space, around an axis passing through the origin, is represented by a quaternion $e^{\mathbf{u}\theta}$, where $\mathbf{u}$ is the imaginary quaternion that correspond to the unit vector oriented along the axis of rotation. So we have the correspondence: $$ \vec{w}=R_{\mathbf{u},\theta} \; \vec{v} \quad \longleftrightarrow \quad \mathbf{w}= e^{\mathbf{u}\theta/2}\mathbf{v}e^{-\mathbf{u}\theta/2} $$
( see Quaternions vs Axis angle)
For example, given the rotation around the axis passing through the origin and the point of coordinates $(1,1,1)$ and angle $\theta=\pi/2$, it is easy to write the corresponding quaternion: $$ e^{\frac{\theta}{4}\frac{ (\mathbf{i}+\mathbf{j}+\mathbf{k})}{\sqrt{3}}} $$ If You don't know how calculate quaternion exponentials you can see my answer in Exponential Function of Quaternion - Derivation.
This is called the decomposition into Euler angles. The problem is that the result depends on which order you're planning to do the x-, y-, and z-rotations in.
Here's the drill, though:
Step 1: Convert your rotation to a $3 \times 3$ orthogonal matrix using Rodrigues' formula. This is well-detailed on Wikipedia and elsewhere, so I won't go into it here.
Step 2: Your matrix $M$ has three columns, $u_1, u_2, u_3$. Let's say you want to write $$ M = R_x R_y R_z $$ where the $R$s are rotations (by varying amounts) around the three axes. We could then write $$ R_y^{-1}R_x^{-1}M = R_z $$ which, since a rotation about $z$ has interesting entries only in its upper left $2 \times 2$ block, means that we need $$ R_y^{-1}R_x^{-1}u_3 = e_3 $$ where $e_3$ is the vector $\begin{bmatrix}0\\0\\1\end{bmatrix}$.
Let's make that happen. First, we'll find an $x$-rotation that "kills off" the first entry of $u_3$.
Case 1: Both the first and last entries of $u_3$ are 0. In this case, perform no rotation about the $y$ axis, and if the second entry is 1, pick $R_x$ to rotate about $x$ by $90$ degrees; if the second entry is $-1$, pick $R_x$ to rotate about $x$ by $-90$ degrees. With these choices, $R_x^{-1}u_3$ will be $e_3$.
Case 2: At least one of these entries is nonzero. Let's call them $s$ and $c$. Let $\alpha = \text{atan2}(c, s)$, and let $R_y$ be rotation about $y$ by angle $\alpha$. Then $R_y^{-1} u_3$ will have its first entry zero, and its third entry nonzero. Call the second and third entries of $R_y^{-1} u_3$ by the names $a$ and $b$ respectively. Let $\beta = \text{atan2}(b, a)$, and let $R_x$ be rotation about $x$ by angle $\beta$. Then $$ R_x^{-1}R_y^{-1} u_3 = e_3. $$ That means that $$ R_x^{-1}R_y^{-1} M = \begin{bmatrix} c' & -s' & 0\\ s' & c' & 0\\ 0 & 0 & 1 \end{bmatrix}, $$ and we can write $\gamma = \text{atan2}(c', s')$ and let $R_z$ be rotation about $z$ by angle $\gamma$, and we're done.
It's quite possible that I've made a sign error here somewhere: that one of the angles should be the negative of what I wrote. But if you write a program to implement this, it should be pretty obvious where any error is.