Imagine there is a tetrahedron frustum in sphere with 8 corners of known positions with respect to some origin in space. Also, imagine there is a random unit vector at some random point in the same coordinate system. Is there a way to quickly check if the vector is pointing towards or away form the frustum? Please be advised that I do know how to calculate the intersection between surfaces of this frustum and the line lying on the vector. However, there is the possibility that the vector is pointing away from the frustum even though the line actually intersects the frustum. I am only interested in the case where the vector is pointing towards the frustum.
Thanks,
Suppose you have a point $\vec{x}$ and a unit vector $\hat{n}$. You say that you can calculate the point of intersection $\vec{y}$ between the surfaces of the frustum and the line defined by $\vec{x}$ and $\hat{n}$. If that's the case, then you simply need to take the dot product: $$ q = \hat{n} \cdot (\vec{x} - \vec{y}) $$ The vector $\vec{x} - \vec{y}$ points from $\vec{y}$ to $\vec{x}$, i.e., away from the frustum. If $q$ is positive, then that means that $\vec{n}$ also points away from the frustum. If $q$ is negative, then $\vec{n}$ points in the opposite direction as $\vec{x} - \vec{y}$, i.e., towards the frustum.
Note that it is possible that a line defined by $\vec{x}$ and $\hat{n}$ does not intersect your frustum at all, but in that case $\vec{y}$ simply doesn't exist. Computationally, you would just need to check which faces your line intersects (there will be at most two), and then calculate the intersections $\vec{y}$ and the corresponding $q$ values for each one.