I am given the problem
Find a necessary and sufficient condition for a point to lie inside or on the tetrahedron formed by the position vectors O ,a, b, c.
In an earlier part of the question I was asked to find an equation of the plane passing through a,b and O which is r . (axb)=0, and then state a condition that, for a point above or below the plane, the point will be on the same side of the plane as a vector c. For this part I considered that the normal to plae on a particular side is axb, so for r and c on the same side it must be that r.(axb) and c.(axb) are the same sign. I am also given that a.(bxc)>0, so a condition must be that r.(axb)>0
In the next part of the question I am asked to show that the equation of a plane through a,b and c is
r.(axb + bxc + cxa)=a.(bxc)
which was simple enough to do.
I have a feeling that these parts are needed for the question I asked at the beginning, which is why I put them here.
Now my first thought was that if I gave a condition for a point being on or above each of the faces, then it must be contained within the tetrahedron. Using this approach, for the faces passing through the origin I get
1. r.(axb)$\geq$0
2. r.(bxc)$\geq$0
3. r.(cxa)$\geq$0
Now for the final face which does not pass through the orgin I am not sure, and even if my approach is correct I am pretty sure it is not the most elegant way to think about this; I considered the position vector a as a new 'origin'. I know that the normal to this plane is (axb + bxc + cxa) and I need a point which lies on the same side of this plane as the actual origin, which is a position vector -a from a
So using the same approach as before I need
(r-a).(axb + bxc + cxa) to be the same sign as (-a).(axb + bxc + cxa)=-a.(bxc). So I need (r-a).(axb + bxc + cxa) to be negative, meaning
4. r.(axb + bxc + cxa)$\leq$a.(bxc)
So I am not sure if this is at all correct, and i'm pretty sure there is a better way to think about this. I have come across something like this before and I think the linear independence of the vectors was used? Perhaps I could say that r= pa+qb+sc where p+q+s=1? I am not sure about this though. In some ways it seems to make sense, but it is not at all intuitive. And also somehow this has only one (or two?) conditions, whereas I had four before! So I don't really understand if either of these answers is correct. Unfortunately I don't seem to have a very good intuition when it comes to geometry.
This reminds me of an old ProjectEuler problem I solved years ago: determine whether a point lies inside of a triangle. The following necessary and sufficient condition is straightforward but a little computation-heavy. Consider the graphic:
The red dot represents the point in question. Suppose that, for each vertex of the triangle, we connect a segment from that vertex to the point. Doing this creates three new triangles. Each of these triangles shares one of its sides with the original triangle and has as its other sides two of the freshly created segments. I have used green to highlight one of these three triangles.
Now consider the sum of the areas of these three triangles relative to the area of the original triangle. What is true if the point lies inside? What is true when the point lies outside? And what happens if the point lies on an edge of the original triangle?
We can compute the length of every segment in each of these figures by using the Pythagorean theorem. Next, we can thank the Hero of Alexandria for providing us a formula for the area of a triangle given only its side lengths.
This strategy can be adapted for the $\mathbb{R}^3$ case. Again, we can find the lengths of the sides of all the relevant tetrahedrons, and, conveniently, a generalization of Hero's formula exists for finding the volume of a tetrahedron given the lengths of its sides.