Check if a line on a plane separates two given points on the plane

Question

Let $P_1$ and $P_2$ be two points on a 3D plane. Let $A$ and $B$ be two points on the same plane, which define a line on said plane.

How can I check if $P_1$ and $P_2$ are on the same side of the line, or on opposite sides?

Answer 1

Let the line $AB$ be $\underline{r}=\underline{a}+\lambda\underline{b}$ where $\underline{a}$ is the position vector of $A$ and let $\underline{n}$ be the normal to the plane.

The signed distance from $P_1$ is $$\overrightarrow{AP_1}\cdot\frac{\underline{b}\times\underline{n}}{|\underline{b}\times\underline{n}|}$$

the corresponding calculation for $P_2$ will be of opposite sign if the points $P_1$ and $P_2$ are on opposite sides of the lines, otherwise they will have the same sign.

Answer 2

One way would be to consider the line $\lambda P_1 + (1 - \lambda P_2)$ for $\lambda \in (0, 1)$, and see if this line intersects the line defined by $A$ and $B$. If it does intersect, they are on opposite sides, and if it doesn't, they are on the same side. I'll leave it to you to decide how to count the situation when $P_1$ or $P_2$ lie on the line defined by $A$ and $B$.

Answer 3

Compute the cross products $(P_1-A)\times(B-A)$ and $(P_2-A)\times(B-A)$. If their components have the same pattern of signs, then they are on the same side of the line.