Given a set of faces and one active face I need to select all faces that share an edge (in physical space, edges are not connected) whose normals flow outwards.
My initial thoughts were to compare the dot product of the normal of the active face with the normal of each face in the set of faces. And this seemed to work for the majority of cases; except when the normal of one of the faces is pointing inward of the other (see A in illustration).
How can I compare the normals and/or faces to eliminate case A from my selection?
It is not enough to compare the directions of the normals. You have to compare the directions of the edges in the list of points for each face. If the points on the common edge are in the same sequence in both faces, the normals are opposite. If they are in the opposite sequence, the normals are the same.