Why require a normal subgroup to be also a subgroup, in the definition?

Question

Let $G$ be a group, $H\subset G$ (but not necessarily $H \leq G$!) and, for all $g\in G$, $gHg^{-1}\subset H$. Is it true that $H$ is a subgroup of $G$? (And if it happens to be true, why to require a normal subgroup to be also a subgroup, in the definition?)

p.s. What follows is the contest from where this question originated (which can be skipped). This existential doubt came to me after reading a proof of the normality of the center $Z(G)$ of a group $G$. From the textbook:

Note that if $z\in Z(G)$, $gzg^{-1}=z$ for any $g\in G$. But then $aZ(G)a^{-1}=Z(G)$ given any $a\in G$. $\square$

As far as I can see, the author does not prove that $Z(G)\leq G$ (but this is evident, and maybe this is the reason why such a check is happily skipped). This made me wonder if there a more subtle reason to not explicitly write down it.

Answer 1

If $G$ is abelian, then $gHg^{-1} = H$ for all $H \subset G$ and $g \in G$. Thus, any subset $H$ that is not a subgroup provides a counterexample, when $G$ is abelian.

Answer 2

Let $H$ be the empty set. It is not a subgroup, but satisfies $gHg^{-1}=H$ (both are empty sets).

Edit: You want to make sure a "normal subset" (a subset $H$ satisfying $gHg^{-1}=H$) is a subgroup, because by the time you see quotient group, you will see that normal subgroups are the only objects over which you can take quotient of $G$.