I am trying to show that $G:=\langle a,b \mid a^2, b^4, (ab)^3 \rangle$ is of order at most 24. (To prove that it is a presentation of $S_4$.)
I thought of using the homomorphism $f$ sending $a$ to $(1 2)$ and $b$ to $(1 2 3 4)$. The image of this homomorphism generates $S_4$, and it satisfies the relations of $G$.
Hence the homomorphism $f$ extends to a homomorphism $f’$ making the following diagram commute
Claim: $f’$ is an isomorphism to its image $f’(G)=S_4$.
Since $f’$ is surjective and its image has order $24$, it suffices to show that $|G|\le 24$. Now, any word in $G$ in $a$ and $b$ can be written in the form $a^{m_1} b^{n_1} a^{m_2} b^{n_2} a^{m_3} b^{n_3}$, or $b^{n_1} a^{m_1} b^{n_2} a^{m_2} b^{n_3} a^{m_3} $ where $m_i \in \{0,1\}$ and $n_i\in \{0,1,2,3\}$.
Now there is $2(2^3\times 4^3)=2(512)=1024$ elements, but since $(ab)^3=1$, then $G$’s elements should be way less than $1024$, but I am not sure how to show that. Any hint would be appreciated.
This solution results from carrying a coset enumeration by hand and then translating it back into standard mathematical language.
Let $H = \langle b \rangle$. We will prove that $$G = H \cup Ha \cup Hab \cup Haba \cup Hab^2 \cup Hab^2a.$$ This shows that $|G:H| \le 6$ and since $b^4=1 \Rightarrow |H| \le 4$, this gives $|G| \le 24$, as required.
Denote the union of these six cosets of $H$ by $X$. To show that $X=G$, it is enough to show that $xg \in X$ for all $x \in X$ andd $g \in G$ and, by an easy inductive argument, it is enough to show that $xa \in X$ and $xb \in X$, for $x \in \{ 1,a,ab,aba,ab^2,ab^2a \}$. So we'll do that!
$1a \in Ha$, $1b \in H$.
$aa=1 \in H$, $ab \in Hab$.
$aba \in Haba$, $abb \in Hab^2$.
$abaa \in Hab$, $abab = b^{-1}a \in Ha$.
$ab^2a \in Hab^2a$, $ab^2b = ab^{-1} = baba \in Haba$.
$ab^2a^2 \in Hab^2$, $ab^2ab = (aba)(abab) = b^{-1}ab^{-1}b^{-1}a=b^{-1}ab^2a \in Hab^2a$.