I'm not sure if my proof works here,
$\Rightarrow$
Given that $gH \leq G$,
$gH$ is a left coset of $H$. Now, the only way to make sure that $gH \leq G$ is that if $gH = H$ (since we already know $H \leq G$), in which case $g$ must be in $H$.
$\Leftarrow$
Given that $g \in H$ and that $H \leq G$,
we know that $g * h \in H$ for all $h \in H$. In other words, $gH = H$. And therefore $H = gH \leq G$.
Why? And how is that related to the fact that $H\leq G$? That's not enough.
The proper explanation would be: assume that $gH$ is a subgroup of $G$. In particular the neutral element $e\in gH$, i.e. $e=gh$ for some $h\in H$. Therefore $g$ is the inverse of $h$ and thus $g\in H$.
The other implication looks fine to me.