How to check if point is within a rectangle on a plane in 3d space

Question

Please refer to this image for this question->

I have a 3d bounded box (in green).

I also have a 3d line (in red)

I know the points a, b, c, d, e. They are points in space with x, y, z, coordinates.

I also know that a, b, c, d, and e, lie on the same plane. i.e. point a is the intersection between the plane of (b, c, d , e) with the red line.

The box is not axis aligned.

Now, what i want to know is, how can i calculate whether the point a, lies inside of (b, c, d, e) box? Obviously it doesn't in this case, but how can i calculate this?

They are on the same plane, so it should be a 2d problem, but all my coordinates are in 3d so i'm not sure how to do it. Can someone help?

This is not homework, i am doing some hobby game programming.

Answer 1

If $b,c,d,e$ are a rectangle and $a$ is coplanar with them, you need only check that $\langle b, c-b\rangle\le \langle a, c-b\rangle\le \langle c, c-b\rangle$ and $\langle b, e-b\rangle\le \langle a, e-b\rangle\le \langle e, e-b\rangle$ (where $\langle,\rangle$ denotes scalar product).

Answer 2

Hint: one way to transform your 3D coordinates to 2D ones, for the purpose of this question.

Set any point on the given plane to be the origin, e.g. I choose B.

$$\begin{align} \mathbb{a'} =& \mathbb{a}-\mathbb{b}\\ \mathbb{b'} =& \mathbb{b}-\mathbb{b} = 0\\ \mathbb{c'} =& \mathbb{c}-\mathbb{b}\\ \mathbb{d'} =& \mathbb{d}-\mathbb{b}\\ \mathbb{e'} =& \mathbb{e}-\mathbb{b}\\ \end{align}$$

Now, since BCDE is a rectangle, $\mathbb{c'}$ and $\mathbb{e'}$ are orthogonal to each other. View $\{\mathbb{c'},\mathbb{e'}\}$ as the basis of the plane.

And since A is on the same plane as BCDE, you can write $\mathbb{a'} = p\mathbb{c'} + q\mathbb{e'}$ form. Now, iff $p$ and $q$ are both within 0 and 1, point A is in BCDE.