Normal Matrix - Is this a valid method of calculation?

Question

Apologies in advance if I have posted unnecessarily. I posted this question on stack overflow but it seems it is more a maths question, hence the re-post here...

I have almost finished coding a 3D graphics engine for a small device, but I have a question regarding normal transforms. The established method to transform object normals using the model matrix is to take the upper-left 3x3 part of the model matrix (M). Then if the object scale is non-uniform, calculate the inverse-transpose such that the normal matrix (N) = transpose(inverse(M)).

If you look at https://stackoverflow.com/questions/27600045/the-correct-way-to-calculate-normal-matrix you will see that Reto Koradi has an apparently much simpler method to calculate the normal matrix, as summarised below:

Calculating the normal matrix yourself is actually quite easy. If the original matrix is written as consisting of row vectors:

    [ r0 ]
M = [ r1 ]
    [ r2 ]

The normal matrix can be calculated by replacing each row by the cross product of the other two rows:

    [ r1 x r2 ]
N = [ r2 x r0 ]
    [ r0 x r1 ]

This neglects a constant factor (which is 1 divided by the determinant). But often times you'll have to re-normalize the transformed normals anyway, so the constant factor does not matter.

I have implemented this method, and it seems to work using some simple testing but I cannot prove it works through theory. Is this method just the same as a 'degenerated' inverse-transpose without the determinant factor, or is it a different and simpler method (with or without flaws)?

I would appreciate any answers the community could give as I have to make sure my normal calculations are always correct!

Answer 1

The formula is sound to me. It relies on the properties of the so called scalar triple product (sometimes also called mixed product).

In short, the product $\mathbf{a}\cdot(\mathbf{b}\times \mathbf{c})$ is zero whenever two of the vectors are equal. Moreover, rotating the vectors (but keeping the same relative order, on the clock) does not change the result. Therefore, in your case

$$ \mathbf{r}_0\cdot(\mathbf{r}_1\times \mathbf{r}_2) = \mathbf{r}_2\cdot(\mathbf{r}_0\times \mathbf{r}_1) = \mathbf{r}_1\cdot(\mathbf{r}_2\times \mathbf{r}_0) $$

So the diagonal entries of $M\cdot N^t$ are the same, and the off-diagonals are all zeros.