How do I find the euler angles if I already have start and ending vector?

Question

I have two orthonormal bases and I need to find the rotation angle over every axes to go from the first to the second one. These are my base vectors:

$$ E_1 = \begin{bmatrix} -0.7969 & 0.1778 & 0.5774\\ 0.2445 & -0.7790 & 0.5774\\ 0.5524 & 0.6012 & 0.5774 \end{bmatrix} $$

$$ E_2 = \begin{bmatrix} -0.7976 & 0.2282 & 0.5583\\ 0.2187 & -0.7533 & 0.6203\\ 0.5621 & 0.6168 & 0.5509 \end{bmatrix} $$

Answer 1

You are dealing with a numerical problem, so to start with, it might be that turning the matrix $E_1$ into $E_2$ cannot be done exactly with a rotation, this will happen if any of the two matrices is not orthonormal (Note1). If they are not orthonormal, you might want to orthonormalize them (Note2). Alternatively, you might just orthonormalize the rotation matrix I compute below.

To rotate $E_1$ to $E_2$ we need to find the matrix $R$ such that $E_2=RE_1$, so we get $R=E_1E^{-1}_2$. So to compute $R$, first find the inverse of $E^{-1}_2$ and multiply it by $E_1$ in the correct order. Now you can either orthonormalize $R$ as I suggested, or proceed to extract your angles from it directly, but then you have to clamp your values between $-1$ and $1$ where necessary (which you might have to do anyway)< to provide correct input to the trigonometric functions.

To extract the angles that rotate around x,y then z, you can apply the following algorithm to the matrix $R= \left( \begin{array}{ccc} r_{00} & r_{01} & r_{02} \\ r_{10} & r_{11} & r_{12} \\ r_{20} & r_{21} & r_{22} \end{array} \right) $:

if (r02 < +1)
{
 if (r02 > -1)
 {
  thetaY = asin(r02);
  thetaX = atan2(-r12,r22);
  thetaZ = atan2(-r01,r00);
 }
 else // r02 = -1
 {
  // Not a unique solution: thetaZ - thetaX = atan2(r10,r11)
  thetaY = -PI/2;
  thetaX = -atan2(r10,r11);
  thetaZ = 0;
 }
}
else // r02 = +1
{
 // Not a unique solution: thetaZ + thetaX = atan2(r10,r11)
 thetaY = +PI/2;
 thetaX = atan2(r10,r11);
 thetaZ = 0;
}

(For more details on this part see: http://www.geometrictools.com/Documentation/EulerAngles.pdf)

Once you are done, you might want to apply your angular rotations to $E_1$ to see how far your result is from $E_2$ and get an idea about the scale of the numerical issues in your chosen solution.

Note1: To check if a matrix is orthonormal, you can calculate its determinant and compare it to $1$. It will never be exactly $1$, but the further it is from it, the less orthonormal the matrix is.

Note2: Should you choose to orthnormalize the $E$ or $R$ matrices you can either use polar decomposition (http://en.wikipedia.org/wiki/Polar_decomposition) of a 3x3 matrix or, less accurately if you can live with it (and your matrices are already almost orthonormal), do the following:
For a matrix $A$ with three column vectors $a_1,a_2,a_3$ first compute the cross product $c_3=a_1 \times a_2$, the compute the cross product $c_3=c_3 \times a_1$, normalize both vectors, and then use the now orthonormal matrix $a_1,c_2,c_3$ instead of the matrix $A$.
Yet another way to normalize a matrix is to convert the matrix to a quaternion, normalize it, and convert it back.