What is the smallest group containing $f$ and $g$ isomorphic to?

Question

Let $f,g : A=\mathbb{R}\setminus \{0,1\} \to \mathbb{R}$ be defined as:

$$f(x)=\frac{1}{x}, \ g(x)=\frac{x-1}{x} $$

The smallest group of functions from $A$ to $\mathbb{R}$ containing $f,g$ under composition is isomorphic to which group?

I tried a few things like finding the inverse of $f,g, f \circ g$, etc. This just got me to suspect that it is possibly of infinite order.

Answer 1

You probably mean $A = \mathbb R \setminus \{0,1\}$. In this case, $f$ and $g$ define functions $A \to A$ and so it makes sense to talk about the group they generate.

We get $$ f^2 = id, \quad g^3 = id, \quad gf = fg^2 $$ Therefore, it's the dihedral group $D_{6}$, the group of symmetries of a regular triangle, aka as $S_3$.

Answer 2

Rather than considering these as functions on $\mathbb{R} \setminus \{0, 1\}$, we can extend them to functions defined on $\newcommand{\P}{\mathbb{P}} \newcommand{\R}{\mathbb{R}} \P^1(\R) = \R \cup \{\infty\}$. We then see that the group $G$ generated by $f$ and $g$ acts on the three points $0,1, \infty$ as permutations: \begin{align*} f: 0 &\mapsto \infty\\ 1 &\mapsto 1\\ \infty &\mapsto 0 \end{align*} \begin{align*} g: 0 &\mapsto \infty\\ \infty &\mapsto 1\\ 1 &\mapsto 0 \end{align*} so $f$ corresponds to the transposition $(0, \infty)$ and $g$ to the $3$-cycle $(0, \infty, 1)$. These two permutations generate the symmetric group $S_3$, and since a Möbius transformation is determined by where it sends any $3$ points, then $G \cong S_3$.

As a note, the $\P^1(\R)$ is the boundary of the completed complex upper half-plane $\mathcal{H}^*$. The automorphism group of $\mathcal{H}^*$ is $\mathrm{PSL}_2(\R)$, which explains the small error in lhf's answer pointed out by PJK and Hagen von Eitzen: $\mathrm{PSL}_2(\R) = \mathrm{SL}_2(\R)/\{\pm I\}$, so $-I$ acts by the identity as well.