I have been doing some work to analyze sets of 5 points in the general configuration.
[![5 points in 3 space][1]][1]
[1]: https://i.stack.imgur.com/ln51Z.jpg
One question I am addressing is whether or not points A, B, E, and D are in the same plane. I am calculating this by the following method.
- I have vectors vBA, vBD, and vBE already calculated.
- vectors vBA and vBD form the plane pABD
- vectors vBA and vBE form the plane pABE
- planes pABD and pABE intersect along the vector vBA
I can calculate the angle at which pABD and pABE intersect along vBA by taking the vector rejection of vBD against vBA and the vector rejection of vBE against vBA and then calculating the angle between the rejection vectors.
Sine the rejection of vBD against vBA lies in the plane pABD and is perpendicular to vBA, the rejection of vBE against vBA lies in the plane pABE and is perpendicular to vBA, and both rejection vectors have their tail at the point B, the angle between these rejection vectors should be the angle at which the planes intersect.
I also calculate the plane intersection angles of pABE & pDBE along vBE and pEBA & pDBA along vBD. If all three of these plane intersection angles are equal to pi, then the points B,A,D,E are all in the same plane.
I originally tried this by calculating the cross products of the the vector combinations BAxBD, BAxBE, and BExBD. I then took that angles between each pair of cross product vectors and if the angles were equal to pi or 0, then the points were all in the same plane.
In programming, the concept of "equal to" is imprecise. It is difficult to evaluate if a real number is equal to 0, let alone pi. Realistically, you can say if it falls in some range around a number. At the moment, I am using 1x10^-9 as a tolerance, meaning that I call a value equal to pi if the value falls in the range of pi +/- 1x10^-9 inclusive. I found that with the cross product approach, I had difficulty creating real world examples (not integers) of 4 points in the same plane where all three cross product vectors were equal to pi or 0 given the above tolerance.
By using the plane intersection by vector rejections, there are fewer computational steps. When comparing the plane intersection angles, I am not going the last step to convert to radians but just taking the cosine as the dot product of the two vectors divided by the product of the vector magnitudes. This way I just have to ask if the plane intersection cosine is equal to -1. With this method, my real world examples are within 2x10^-16 of -1.
This is one such example,
id x y z
B -0.394666 -0.0990807 -0.610038
A 1.90877 1.16538 -0.268284
D -0.672164 -0.83449 -1.06511
E -0.707941588 0.02009757 -0.449844072
C -15.359866 -23.1923807 21.479662
ABD_EBD plane intersection cosine = -1 ABE_DBE plane intersection cosine = -1.000000000000000222 EBA_DBA plane intersection cosine = -0.999999999999999889
I have some questions regarding this methodology.
- Is the method of using plane intersection angles valid and supportable?
- Am I right that I can just evaluate if the cosine of the rejection vectors is -1 instead of taking the arc cosine and evaluating if the angle is equal to pi?
- What is a reasonable tolerance to use when evaluating equivalencies given that I am using double precision on a 64-bit operating system? Can someone with experience tell me if 1x10^-9 is too small or too large?
- please let me know if you want me to post any of the formulas I am using to calculate the different steps described above? Also let me know if more pictures are required.
Thanks for the advice,
LMH

"One question I am addressing is whether or not points A, B, E, and D are in the same plane."
You can do this much simpler than the method you are suggesting.
Method 1: Construct a matrix out of vBA, vBD, and vBE. Calculate its determinant. If it is zero (within tolerance) the vectors are coplanar and point E lies in plane ABD.
Maybe easier to understand, method 2:
$$ \vec {BE} = s \vec {BA} + t \vec {BD} $$
Dot this with the basis vectors.
$$ ( \vec {BA} \cdot \vec {BE} ) = s ( \vec {BA} \cdot \vec {BA} ) + t ( \vec {BA} \cdot \vec {BD} ) $$
$$ ( \vec {BD} \cdot \vec {BE} ) = s ( \vec {BD} \cdot \vec {BA} ) + t ( \vec {BD} \cdot \vec {BD} ) $$
( Note: $ ( \vec {BD} \cdot \vec {BA} ) = ( \vec {BA} \cdot \vec {BD} ) $. )
Solve the two equations two unknowns for s and t. Plug them into the plane equation to get the projection of $\vec {BE}$
$$ \vec {P} = s \vec {BA} + t \vec {BD} $$
Subtract the projection from the vector to get the normal displacement from the plane:
$$ \vec {D} = \vec {BE} - \vec {P} $$
The length of $ \vec {D} $ is how far the point E is from the ABD plane.
Hope this helps.
Ced
P.S. I just saw amd's comment. This is basically that.
Followup:
On precision. It doesn't matter how big your system's cpu registers are, what matters is the size of the number being used. A standard "double" is an eight byte floating point which is good to about 15 significant decimal digits. So 10^(-9) is huge in comparison. What really matters more is the nature of your values. If they are real world values measured to say, three significant digits, then that is the tolerance you should be working with.