I have 4 points and need to determine which pairs of these points represent the diagonals.
In other words, I am trying to triangulate a quadrilateral.
I realize that triangulation of polygons is a well documented subject, but I'm looking for a short algorithm that I can implement in a few lines. Hopefully since I only have quadrilaterals, I can avoid importing a whole triangulation library.
There are three possible groupings of points into pairs that could form the two diagonals. For each of them, calculate the sum of the edge lengths:
\begin{align*} (p_1, p_2), (p_3, p_4) &:& \|p_2-p_1\|+\|p_4-p_3\|\\ (p_1, p_3), (p_2, p_4) &:& \|p_3-p_1\|+\|p_4-p_2\|\\ (p_1, p_4), (p_2, p_3) &:& \|p_4-p_1\|+\|p_3-p_2\| \end{align*} whichever has the highest sum on the RHS is the diagonals. This works in any dimension provided the four points are coplanar.