$G$ group, $H \unlhd G$; prove/disprove $gC_G(h) \cap H \ne \emptyset, \forall h \in H, g \in G$.

Question

Let $G$ be a (possibly infinite) group and $H \unlhd G$. For a given $h \in H$, $G$ is partitioned into the set of (e.g. left) cosets of $C_G(h)$ in $G$. I would like to prove/disprove the following

Claim: $gC_G(h) \cap H \ne \emptyset, \forall h \in H, g \in G$.

It holds for $g \in H$, since then $gh \in H \cap gC_G(h), \forall h \in H$. If $g \notin H$, we can say that $\exists g' \in G \setminus H$ such that $g \in Hg'$, but I can't go farther than that.

Answer 1

This property holds if and only if $G$ fixes the conjugacy classes of $H$ under conjugation.

First suppose $G$ fixes the conjugacy classes of $H$ under conjugation. This means that for any $g\in G$ and $h\in H$ we have $h^{g^{-1}}=h^x$ for some $x\in H$. This gives $g^{-1}x^{-1}\in C_G(h)$ so $x^{-1}=gg^{-1}x^{-1}\in gC_G(h)\cap H$.

Conversely suppose $gC_G(h)\cap H\ne\emptyset$ for all $h\in H$. Then for any $h\in H$ we have $g^{-1}c=x$ for some $c\in C_G(h)$ and $x\in H$, so $g=cx^{-1}$ and $h^g=h^{cx^{-1}}=h^{x^{-1}}$, so $G$ fixes the conjugacy classes of $H$.

In particular the property does not hold for any abelian $H$ not contained in the center of $G$.