Let $G$ be a non-discrete Lie Group acting properly on a Riemannian manifold $M$ by its isometries. When $G$ is discrete, there are plenty of literature on Fundamendal Domains (FD), but a quick literature search shows none on FD when $G$ is not discrete, and is say a Lie Group of manifold dimension at least one.
In what follows, $d_M$ is the metric on $M$ induced by the Riemannian metric $g$ on $M.$
My first question is:
- What's the reason behind the fact that we don't see any literature on FD when $G$ is non-discrete?
Also, a very common FD used when $G$ is discrete is the Dirichlet Fundamental Domain $F(p_0)$ associated with the point $p_0\in M$, see for example: this link, this link, this link, when $M:=\mathbb{H}^2, G$ a discrete subgroup of its isometry group, given by:
$$F(p_0):=\{p\in M: d_M(p,p_0)\le d_M(p, g.p_0)\forall g \in G\}$$
I also consulted this MO post, but in this excellent answer given by Misha, the assumption remains the same: $G$ is discrete. Before asking my next questions, let me copy and paste the definition from Misha's answer there, so that this question is self-contained.
Definition. Let $M$ be a Riemannian manifold. Let $G\times M\to M$ be a proper action of a discrete group. An open subset $D\subset M$ is a fundamental domain for the action if:
The interior of the closure of $D$ equals $D$.
$G$-orbit of $cl(D)$ equals $M$.
$g(D)\cap D=\emptyset$ for all $g\in G -\{1\}$.
The collection of subsets $\{g cl(D): g\in G\}$ is locally finite.
My subsequent questions are:
- Where exactly is the discreteness assumption playing a role to guarantee the existence of a fundamental domain, or proving that $F(p_0)$ above (or its interior, depending on the definition you choose to use) is a fundamental domain? Will it not respect the property 4) above, but will respect the other three if we assume $G$ is non-discrete?
- What problem will we run into if we work with the subset $F(p_0):=\{p\in M: d_M(p,p_0)\le d_M(p, g.p_0)\forall g \in G\}?$ Will it fail to be a fundamental domain if $G$ is not discrete? A very related question was earlier asked by another author here, but received no answer.
Let's suppose that you reword the definition of a fundamental domain by throwing away the discreteness requirement: just replace the phrase proper action of a discrete group by the phrase proper action of a topological group.
For the proof, assuming that $G$ is not discrete, I'll argue to a contradiction. Because of the failure of discreteness, there exists a sequence $g_i \in G$ of nonidentity elements of $G$, such that $g_i \ne g_i$ if $i \ne j$ and such that $g_i$ that converges to the identity element of $G$.
Pick a point $x \in D$.
By item 4, $x$ has an open neighborhood $U$ such that the set $$S_U = \{g \in G \mid g(cl(D)) \cap U \ne \emptyset\} $$ is finite. But since $g_i$ converges the identity, it follows that $g_i(x)$ converges to $x$, and so there exists $I$ such that $i \ge I \implies g_i(x) \in U$. But $x \in D \in cl(D)$, and so if $i \ge I$ then $g_i(D) \cap U \ne \emptyset$. The set $S_U$ is therefore infinite, which is a contradiction.
Notice that the only portion of the definition I used is item 4. That item all alone is what enforces discreteness.
This answers your question 2. It also answers your question 1; there is little literature on non-existent objects. It also answers your question 3, because what my proof shows is that for any nonempty subset $D \subset M$, if $G$ is not discrete then the set $\{g(cl(D)) \mid g \in G\}$ is not locally finite.