Let $X$ a G-set and $x \in X$. Show that for any $g\in G$, the stabiliser of $g.x$ is the subgroup $gGxg^{-1}$.
I found this question in the book A Course in Group Theory of J. Humphreys. I was trying solve, but this question no make sense for me. Look, $x \in X$ but $X$ is not necessarily subset of $G$, then no make sense say that $gGxg^{-1} \leq G$. For this make some sense $X$ must be subset of $G$, in this case $G=gGxg^{-1}$ that no have grace.
It is likely that I'm wrong, but I do not know why.
Your mistake comes from (in my opinion) a bad notation (but it is widely used unfortunately). There are two subsets at stake when you give some $x\in X$ :
$$Gx=G.x:=\{g.x|g\in G\}\text{ which is the orbit of }x\text{ under the action of } G $$
$$\text{ and }$$
$$G_x=Stab_G(x):=\{g\in G|g.x=x\}\text{ the stabilizer of } x\text{ under the action of } G$$
Hence we see that $Gx$ is a subset of $X$ whereas $G_x$ is a subgroup of $G$.
Here you have mistaken $G_x$ for $Gx$. I suggest you use $Stab_G(x)$ instead of $G_x$ for the stabilizer so you won't do this mistake anymore.
Now for the proof that $gG_xg^{-1}=G_{g.x}$ it comes from the following equivalences :
$$h\in G_{g.x}\Leftrightarrow hg.x=g.x\Leftrightarrow g^{-1}hg.x=x\Leftrightarrow g^{-1}hg\in G_x\Leftrightarrow h\in gG_xg^{-1} $$