A friend asked me to help him prove that the topological group $\mathrm{Homeo}(0,1)$ (homeomorphism of $(0,1)$ with the compact open topology) is Polish (that is, separable and completely metrizable).
Denote $X=(0,1)$.
We know that for a compact and metrizable space $Y$, the group $\mathrm{Homeo}(Y)$ is Polish. We tried to apply a similar proof to the case of $X$.
The problem starts with separability. $\mathrm{Homeo}(Y)$ inherits its separability from $C(Y,Y)$ (the space of all continuous function). But when we loose compactness, we can't see how one can show that $C(X,X)$ is separable.
So we decided to ask for help:
for $X=(0,1)$
Is $C(X,X)$ separable?
Is $\mathrm{Homeo}(X)$ Polish?
Hints and/or counterexamples will be highly appreciated!
Thank you!
Edit
We thought to use the theorem of Birkhoff-Kakutani, according to which in order to prove metrizability it is enough to show that the identity element has a countable nbhd basis. But it only gives us metrizability, not complete metrizability. And even then, suppose that we know that there is some metric that makes this group completely metrizable, what would this imply about this group with the compact open topology?
Since $X$ is a metric space, the compact-open topology on $C(X,X)$ coincides with the topology of compact convergence. With the latter viewpoint, it is in some respects easier to treat.
We choose a normal exhaustion of $X$ by compact sets, say $K_n = \left[3^{-n},1-3^{-n}\right]$ for $n \geqslant 1$, and since $X$ is homeomorphic to $\mathbb{R}$, we see that $C(X,X)$ is homeomorphic to a (closed) subspace of
$$\prod_{n=1}^\infty C(K_n,\mathbb{R}).$$
By Weierstraß' approximation theorem, the space of polynomials is a dense subspace of $C(K_n,\mathbb{R})$, and it is easy to see that the space of polynomials with rational coefficients is then also dense. Hence $C(K_n,\mathbb{R})$ is separable, therefore also the countable product of such spaces, and since it is metrisable, also its subspace (corresponding to) $C(X,X)$. Since $\mathbb{R}$ is complete in the usual metric, we now have the
Proposition: $C(X,X)$ is a Polish space.
So it remains to see whether $\operatorname{Homeo}(X)$ is a $G_\delta$-set in $C(X,X)$, or equivalently some Polish subspace of that, since by Mazurkiewicz' theorem, a subspace of a Polish space is Polish if and only if it is a $G_\delta$.
Lemma: $\overline{\operatorname{Homeo}(X)} = \operatorname{Mon}(X)$, where $\operatorname{Mon}(X)$ is the set of monotonic functions in $C(X,X)$.
Proof: The pointwise limit of a sequence of monotonically nondecreasing functions is monotonically nondecreasing, and similarly for monotonically nonincreasing functions. Hence $\operatorname{Mon}(X)$ is closed. To see that the set of homeomorphism is dense in the monotonic functions, let $f \in \operatorname{Mon}(X)$, without loss of generality let $f$ be nondecreasing. Choose a compact $K \subset X$ - without loss of generality $K$ is an interval - and $\varepsilon > 0$. Since $K$ is compact, there is an $\eta > 0$ such that $f(K) \subset [2\eta,1-2\eta]$. Choose $\eta < \varepsilon$. On the interval $K$, define $g(x) = f(x) + \eta\cdot x$. Then $g(K) \subset [2\eta,1-\eta]$, and $g$ is strictly increasing on $K$. Extend $g$ to a homeomorphism of $X$ by linear interpolation on $(0,\min K)$ and $(\max K,1)$. On $K$, we have $\lvert g(x) - f(x)\rvert = \eta\cdot x < \eta < \varepsilon$, hence $g\in U(f;K,\varepsilon)$, and the set of homeomorphisms is seen to be dense in $\operatorname{Mon}(X)$.
There are two ways in which a function in $\operatorname{Mon}(X)$ can fail to be a homeomorphism, it can be not injective, hence constant on some subinterval of $X$, or it can be not surjective. The first failure mode is easy to deal with:
Lemma: $\operatorname{SMon}(X)$, the set of strictly monotonic functions in $C(X,X)$ is a $G_\delta$ in $\operatorname{Mon}(X)$ (and hence also a $G_\delta$ in $C(X,X)$).
Proof: For any $0 < a < b < 1$, the set $Co(a,b) := \{ f\in C(X,X) : f\lvert_{[a,b]} \equiv \operatorname{const}\}$ is closed. Since there are only countably many pairs of rational numbers, and every non-injective monotonic function is constant on some non-degenerate interval with rational endpoints,
$$\operatorname{SMon}(X) = \bigcap_{\substack{0 < a < b < 1 \\ a,b \in\mathbb{Q}}} \left(\operatorname{Mon}(X)\setminus Co(a,b)\right)$$
is a $G_\delta$ in $\operatorname{Mon}(X)$.
Now it remains to see that $\operatorname{Homeo}(X)$ is a $G_\delta$ in $\operatorname{SMon}(X)$. A strictly monotonic $f \in C(X,X)$ is a homeomorphism if and only if it is surjective, that is if and only if $\inf f(X) = 0$ and $\sup f(X) = 1$.
For any $a \in X$, the sets $A(a) = \{ f\in C(X,X) : \sup f(X) \leqslant a\}$ and $B(a) = \{ f \in C(X,X) : \inf f(X) \geqslant a\}$ are closed. Hence
$$\operatorname{Homeo}(X) = \bigcap_{a \in X\cap\mathbb{Q}} \left(\operatorname{SMon}(X) \setminus \left(A(a) \cup B(a)\right)\right)$$
is a $G_\delta$ in $\operatorname{SMon}(X)$.
Wrapping it up, we have proved the
Theorem: $\operatorname{Homeo}(X)$ is a Polish space in the compact-open topology.
(There's probably a more elegant proof, but this is what I came up with.)