Ear clipping triangulation snip calculation

Question

I have a working code of 2D triangulation of a polygon. This uses the following code to detect if a triangle is actually an ear:

private bool Snip (int u, int v, int w, int n, int[] V) 
{
    var A = m_points[V[u]];
    var B = m_points[V[v]];
    var C = m_points[V[w]];

    if (Mathf.Epsilon > (((B.x - A.x) * (C.y - A.y)) - ((B.y - A.y) * (C.x - A.x))))
        return false;

    for (var p = 0; p < n; p++) 
    {
        if ((p == u) || (p == v) || (p == w))
            continue;

        var P = m_points[V[p]];

        if (InsideTriangle(A, B, C, P))
            return false;
    }

    return true;
}

Now I'd wish to make this a 3D function, since I have to rotate all polygons according to their normal into 2D space. The one confusing part is that I do not understand what this code does:

if (Mathf.Epsilon > (((B.x - A.x) * (C.y - A.y)) - ((B.y - A.y) * (C.x - A.x))))

Thus making it impossible for me to create a 3D variant on it.

Does anyone know what this does?

Answer 1

The expression on the right-hand side of the inequality is computing the determinant of

$$ \begin{bmatrix} B.x-A.x&B.y-A.y \\ C.x-A.x&C.y-A.y \\ \end{bmatrix}, $$ which is twice the (oriented) area of the triangle $ABC$. So the expression determines if the triangle's area is very small (less than Mathf.Epsilon), or if the triangle's vertices are in clockwise order. I suspect the latter is a bug, and the intention was to was to check the area; in that case, the absolute value should be taken before comparing.

For three dimensions, you would presumably want to compute the corresponding determinant for the volume, using the matrix $$ \begin{bmatrix} B.x-A.x&B.y-A.y&B.z-A.z \\ C.x-A.x&C.y-A.y&C.z-A.z \\ D.x-A.x&D.y-A.y&D.z-A.z \end{bmatrix}, $$ or the 4x4 matrix $$ \begin{bmatrix} A.x & A.y & A.z & 1\\ B.x & B.y & B.z & 1\\ C.x & C.y & C.z & 1\\ D.x & D.y & D.z & 1 \end{bmatrix}. $$