Does the rectangle contain the point?

Question

A rectangle is defined by the 4 points ABCD. How can I tell if a given point, (x,y), is in the interior of the rectangle?

My current guess is the following:

• To be inside the rectangle, the point should be between the lines AB and CD, and between the lines AD and BC.
• The equation of line AB (and, similarly, the other lines) can be written as:

$$ \frac{x-x_A}{y-y_A} = \frac{x_B-x_A}{y_B-y_A} $$

• In order to be between line AB and line CD, one of the following must hold:

Either:

$$ \frac{x-x_A}{y-y_A} > \frac{x_B-x_A}{y_B-y_A} and \frac{x-x_D}{y-y_D} < \frac{x_C-x_D}{y_C-y_D} $$

Or vice versa:

$$ \frac{x-x_A}{y-y_A} < \frac{x_B-x_A}{y_B-y_A} and \frac{x-x_D}{y-y_D} > \frac{x_C-x_D}{y_C-y_D} $$

(According to the actual values of A, B, C and D, some of these conditions may be impossible or trivial).

• In order to be between line AD and BC, one of the following must hold:

Either:

$$ \frac{x-x_A}{y-y_A} > \frac{x_D-x_A}{y_D-y_A} and \frac{x-x_B}{y-y_B} < \frac{x_C-x_B}{y_C-y_B} $$

Or vice versa:

$$ \frac{x-x_A}{y-y_A} < \frac{x_D-x_A}{y_D-y_A} and \frac{x-x_B}{y-y_B} > \frac{x_C-x_B}{y_C-y_B} $$

Is this correct? Is there an easier way to calculate if a point is inside a square?

Answer 1

Assuming $ABCD$ are in cyclic order around the rectangle, if $P$ is your point, consider the points as vectors. You want $(A-B)\cdot P$ to be between $(A-B) \cdot A$ and $(A-B) \cdot B$ and $(A-D) \cdot P$ to be between $(A-D)\cdot A$ and $(A-D) \cdot D$.

Answer 2

I suspect that you mean "quadrilateral", not rectangle (i.e. a polygon with 4 sides).

If so, then this is the classical "point in polygon" problem. There are two ways to attack it -- one way is to use ray casting, counting crossings, and the other is to use winding numbers.

You'll find lots of other material if you search for "point in polygon". Here is one link, and here is another one.