In the $x,y,z$ space, I want to check if a point lies within a polygon. The $z$ coordinates for the vertices of the will not change, but will do for the point for which we're checking. How can I check this?
I have attached an image which might make it a bit clearer
The first thing you want to check is that your point, let's call it $P$, lies in the plane of polygon $ABCD$. You can use vectors and cross products for this step. You know that any three points form a plane. You can describe this plane by the normal to the plane. $$\vec n_{PAB}\propto (\vec P-\vec A)\times(\vec P-\vec B)$$ If the cross product is zero, it means that $P$ lies on the $AB$ line, and all you need to check if it is between $A$ and $B$. For that, use scalar product: $$ (\vec P-\vec A)\cdot(\vec P-\vec B)$$ If the scalar product is $0$, $P=A$ or $P=B$. If the scalar product is negative, the points is between the two, otherwise is outside. In this last case, it might still be inside the polygon (for concave polygons). Similarly, you calculate $\vec n_{CD}$. If the point $P$ is in the $ABCD$ plane, the two normal vectors are parallel (or anti parallel). You can check this by doing the cross product one again $$\vec n_{AB}\times\vec n_{CD}$$ When $P$ is in the plane, the cross product is $0$.
The other part of the problem is to check if the point $P$ on the plane is between $ABCD$. If it is, it means that the projection of $P$ on the $xy$ plane is between the projection of $ABCD$ in the $xy$ plane. To solve this problem, there are many possible solution. Here is a possible choice. You can also look at the links on this wikipedia page