Let $\mathbb{P}$ be the set of all points contained in a polygon defined by $n$ vertices in the plane $p_i=(x_i, y_i)$, $i=0,\ldots,n-1$.
Does there exist a continuous bijective map $f:\mathbb{R}\to\mathbb{P}$, and furthermore, is there an efficient algorithm to explicitly compute this map (and its inverse)?
If necessary, you may assume $\mathbb{P}$ is convex. By default, assume that the sides of the polygon are given by lines between consecutive points $p_{i-1}$ and $p_i$ (with the index wrapping around).
As a big bonus, I'd like if the map algorithm was invariant to cyclic rotations of the points. That is to say, if I instead gave points $q_i = p_{i+k}$ for some $k$ (with the index wrapping around), the map should still be essentially the same.
As a simple example, if $\mathbb{P}$ was the unit square, then a map would be $f(x, y) = \left(\Phi(x), \Phi(y)\right)$, where $\Phi$ is the cdf of the normal distribution.
This is just a guess. I believe this map is bijective and piece-wise continuous, but I'm not quite sure if it's actually continuous. It has the advantage of being simple to compute. We will apply the condition that the polygon is convex.
1) For each $\theta \in [0, 2\pi)$, map the point with polar coordinate $(r, \theta)$ to the point $(\tanh(r), \theta)$. This maps the plane to the unit disc.
2) Since the polygon is convex, it's star-shaped with respect to it's center of mass, which is inside of it. Then, for each angle in $[0, 2\pi)$, simply scale the ray in the direction of $\theta$, now reduced to a line segment $[0, 1)$. Now, a ray eminating again with angle $\theta$ wrt to the x-axis from the convex polygon's center of mass will be inside of the polygon for some distance, then outside of it. (Since intersecting again with the polygon's interior violate's convexity). That distance is a little nontrivial to program, since there might be if states relating to the fact the polygon is circular, but up to some edge cases you can basically do binary search on the angle (since each vertex must correspond to a distinct angle $[0, 2\pi)$, to find what two vertices the ray intersects the polygon's boundary between and it's a simple linear equation to compute the intersection point and the distance after that. Once you have that distance, simply scale your line segment in direction $\theta$ by that distance, and that defines your map.
This depends only on the geometry of your polygon, and so is invariant under the ordering of your points, but does make heavy use of convexity.
For your unit square example, the map first maps $(x, y)$ to $(\tanh(r)\cos(\theta), \tanh(r)\sin(\theta))$ where $r = \sqrt{x^2+y^2}$ and $\theta = \operatorname{atan2}(y, x)$, which simplifies to be $\displaystyle\left(\frac{x\tanh\left(\sqrt{x^2+y^2}\right)}{\sqrt{x^2+y^2}}, \frac{y\tanh\left(\sqrt{x^2+y^2}\right)}{\sqrt{x^2+y^2}}\right)$. From here, a little trig tells you that for an angle $\theta$, you have a scale factor by the angle of $\frac{1}{2}\min \left(|\sec (\theta)|, |\csc(\theta)| \right)$. This is where I might be getting myself confused, but I believe everything's okay since this is a continuous map, even though it definitely isn't a differentiable one. So now we simply shift over to the center of mass $(\frac12 , \frac12)$, and scale appropriately, so in the end, $(x, y)$ maps to $\displaystyle\left(\frac12 + \frac{x\tanh\left( \sqrt{x^2+y^2} \right)\sqrt{1 + \left(\frac{\min(|x|, |y|)}{\max(|x|, |y|)}\right)^2}}{2\sqrt{x^2+y^2}}, \frac12 + \frac{y\tanh\left( \sqrt{x^2+y^2} \right)\sqrt{1 + \left(\frac{\min(|x|, |y|)}{\max(|x|, |y|)}\right)^2}}{2\sqrt{x^2+y^2}} \right)$
Also note that smoothness isn't a huge issue for some applications of this. If you have the time, it's not so tough to compute a generalized Weierstrass transform with some small width paramater, giving a smooth result. For less complexity, if you require only a few order of differentiability but continuity will not due (I don't know much about how PyTorch's autograd works, but one might presume you want differentiability), if you can stand the approximation just use like a cubic spline around the spots of the vertices. Or if second order differntiability would be preferable a quintic one. You get the idea.