Given the $4$ starting $x,y$ coordinates of a rectangle and the angle by which it is rotated how to calculate if a point is inside the rectangle

Question

I have a starting rectangle with edges parallel to the axis and its center at the origin. I have all the $x,y$ coordinates of each vertex of the rectangle. Then I rotate the rectangle by an angle around the origin which is the center of the rectangle. I then have an $x,y$ point which I want to check if it is within the rectangle or outside of it. Basically to put it another way is there an equation or inequality equation that tells if a given $x$ and $y$ are within the rotated rectangle.

Answer 1

Since rotation won't change the shape of the rectangle all points inside the initial rectangle remain in the rotated rectangle. Meaning if a point is inside the rectangle initially then it will be in the rectangle upon rotation.

So to check if a point $(x,y)$ is inside the rotated rectangle rotate the point in the opposite direction by the same amount and check if it is in the non rotated rectangle.

Say the initial rectangle has corners $(x_1, y_1)$ and $(x_2, y_2)$. Now we rotate the by angle $\theta$. Then we have rotation matrix $$\begin{bmatrix}\cos \theta & -\sin \theta \\ \sin \theta & \cos\theta\end{bmatrix}$$ For a point $(x,y)$ check if $$\begin{bmatrix}\cos \theta & -\sin \theta \\ \sin \theta & \cos\theta\end{bmatrix}^{-1}\begin{pmatrix}x \\y\end{pmatrix} = \begin{pmatrix}x' \\y'\end{pmatrix}$$ Check $\min(x_1,x_2) \le x'\le\max(x_1,x_2)$ and $\min(y_1,y_2) \le y'\le\max(y_1,y_2)$ $$\begin{pmatrix}x' \\y'\end{pmatrix}=\begin{pmatrix}x\cos\theta-y\sin\theta \\ x\cos\theta+y\sin\theta\end{pmatrix}$$

Answer 2

A more general way, though much less efficient in this case than Piyush Divyanakar's solution, will work for any convex polygon.

The basic subproblem is decide if two points are on the same side of a line. If the equation of the line is $ax+by+c = 0$, substitute the coordinates of each point into the equation of the line.

If the results have the same sign, the points are on the same side of the line; if not, they are on different sides.

If you have a convex polygon (this works in any number of dimensions), the average of all the vertices is inside the polygon.

Therefore, to test if a point is inside the polygon, first get the equations of all the bounding lines, get the center as the average of the vertices, and get the sign when the center is substituted into each bounding line.

Then, to test the point, substitute it in the equation of each bounding line. If the sign differs from the sign when the center is substituted, the point is outside; if the sign is the same for all bounding lines, then the point is inside.

Note: There can be problems when the point is close to or on a bounding line, especially when the computation is done in floating point.