Edited : )
Let $H$ and $K$ be two subgroups of $G$ such that $|H|=|K|$. Suppose that $(|H|,|G:H|)=1$, then $N_H(K)=N_K(H)=H\cap K$.
Hint given:
Let $\pi$ denote the set of the prime divisors of $|H|~(\text{or } |K|)$. For convenience, we may call a natural number "$\pi$-number", if all of its prime divisors of are contained in $\pi$. Try to prove if $H\cap K$ is a proper subgroup of $N_H(K)~(\text{or } N_K(H))$, then $|N_H(K)|$ is also a $\pi$-number dividing $|G|$, which contradicts $(|H|, |G:H|)=1.$
I tried to develop the hint, but found something confusing to me. I think that just without $H\cap K$ being proper, $N_H(K)\leq H$ implies that $N_H(K) $ is certainly a $\pi$-number. But the given hint assumed $H\cap K$ to be proper, how does that help? And moreover, how can I develop the hint? I’m rather confused now... : (
PS: I must have missed certain things, but what are they? And how to solve this problem? More detail is in need, please. I'll be grateful if you could provide me a possible answer! Thanks!
A subgroup $H$ with the set of prime divisors $\pi$ is called a Hall $\pi$-subgroup if $(|H|,[G:H])=1$. In particular Sylow $p$-subgroups are Hall subgroups. Recall the Sylow theorem stating that if a Sylow $p$-subgroup $P$ is normal in $G$, then any $p$-subgroup of $P$ is contained in $G$. Same statement applies to normal Hall subgroups. Since $H$ is normal in $N_G(H)$, any $\pi$ subgroup of $N_G(H)$ is contained in $H$. Consider the subgroup $N_K(H)$ of $N_G(H)$. If $N_K(H)$ properly contains $H\cap K$ then there is a prime dividing $[N_K(H):H\cap K]$ which is not in $\pi$. However this prime also divides $|K|$ and hence must be in $\pi$. Contradiction.