Rotating a triangle in 3d specified by the normals

Question

In my surface reconstruction algorithm I need to calculate the right direction of the triangles. The coordinates of the original triangle are known. The original (initial) normal, the new desired normal and the center of triangle were calculated. How could I determine the rotating matrix to get the new coordinates of the triangle?

Answer 1

One approach is to build an orthogonal coordinate frame such that the $x$ axis is in the direction of the initial normal, the plane $xy$ is defined by the initial and desired normals, and $z$ is orthogonal to both (you can find the direction of $z$ as the cross-product of the vectors, and the direction of $y$ will follow).

Now, assuming that all vectors are unit normalized, form a matrix $T$ with the vectors $x,y,z$. If you change the coordinates to the new frame (which is the application of the matrix transpose), the initial vector becomes $(1,0,0)$, and the desired vector $(\cos\theta,\sin\theta,0)$. It is an easy matter to write the 2D rotation matrix, let $R$. After the 2D rotation, you revert to the original frame.

Your $3D$ rotation matrix is

$$TRT^{-1}.$$