How to prove a "sharper Cayley theorem"?

Question

If $H$ is a subgroup of a group $G$, let $X$ designate the set of all the left cosets of $H$ in $G$. For each element $a\in G$, define $\rho_a:X\to X$ as follows: $$\rho_a(xH)=(ax)H$$

1. Prove that each $\rho_a$ is a permutation of $X$
2. Prove that $h:G\to S_X$ defined by $h(a)=\rho_a$ is a homomorphism
3. Prove that the set $\{a\in H:xax^{-1}\in H \: \text{for every}\:x\in G\}$, that is, the set of all elements of $H$ whose conjugates are all in $H$, is the kernel of $h$
4. Prove that if $H$ contains no normal subgroup of $G$ except $\{e\}$, then $G$ is isomorphic to a subgroup of $S_X$

This is given as an exercise called "A sharper Cayley theorem" from "A book of Abstract Algebra" by Charles C. Pinter. Here is my attempt:

1. For $\rho_a$ to be a permutation of $X$, it must be a bijection:

   • $xH=yH\to xh_1=yh_2$ for some $h_1,h_2\in H$ which implies $axh_1=ayh_2$ and $(ax)H=(ay)H\to \rho_a(xH)=\rho_a(yH)$ so $\rho_a$ is well defined

   • Every $xH\in X$ is of the form $xH=\rho_a((a^{-1}x)H)$ so $\rho_a$ is surjective

   • $\rho_a(xH)=\rho_a(yH)\to (ax)H=(ay)H\to axh_1=ayh_2$ for some $h_1,h_2\in H$, so $xh_1=yh_2$ and $xH=yH$ so $\rho_a$ is injective

Now $[\rho_a\circ \rho_b](xH)=\rho_a(\rho_b(xH))=\rho_a((bx)H)=(abx)H=\rho_{ab}(xH)$ and $(\rho_a)^{-1}=\rho_{a^{-1}}$ by the same reason, so all the $\rho_a$ form a group (subgroup of $S_X$) with composition and identity $\rho_e$

2. $h(ab)=\rho_{ab}=\rho_a\circ \rho_b=h(a)\circ h(b)$ so $h$ is a homomorphism
3. This is the part I cannot get through:
   • If $x\in Ker\:h$ then $h(x)$ must be equal to $\rho_e$. $\rho_x(yH)=\rho_e(yH)=yH\to (xy)H=\rho_{xy}(H)=xHyH=yH$, so $xH=H$ and $x\in H$ and $Ker\:h\subset H$
   • If $x\in H$, then $\rho_x(yH)=(xy)H=xHyH$ but $xH=H$ so $\rho_x(yH)=yH$ and $H\subset Ker\:h$, hence $H=Ker\:h$

Where is my mistake? I cannot see where the "conjugates" part plays a role. I think it might be about the homomorphism $h$, because the fact $h(ab)=h(a)h(b)$ which implies $(xy)H=xHyH$ would somehow make sense with the normal subgroup of $H$

Answer 1

You go from $xyH = yH$ to $xH = H$, which is not correct. Note that $aHbH$ is not necessarily equal to $abH$ for non-normal subgroups $H$.

In general, $aH = bH$ specifically when $b^{-1}a \in H$. Thus $xyH = yH$ is equivalent to $y^{-1}xy \in H$. Since $y$ was arbitrary, this shows that an equivalent condition that $\rho_x = \rho_e$ is that all conjugates of $x$ must lie in $x$.