Let $G:=BS(m,n)= \langle a,t \mid ta^mt^{-1}=a^n\rangle$ be a Baumslag-Solitar group (here $m,n\in \Bbb Z\setminus\{\pm 1\})$.
I was trying to determine the normal closure of the element $a\in G$; here, by normal closure I mean $\langle a\rangle^G:=\langle\{gag^{-1}\mid g\in G\}\rangle$.
At first I thought that $\langle a\rangle^G\cong F_\infty$ as the set $\{t^{r}at^{-r}\mid r\in \Bbb Z\}$ is contained in $\langle a\rangle^G$.
But this set is not a free generating set since $a^n=(tat^{-1})^m$.
So I want to know, what is the subgroup $\langle a\rangle^G$ precisely?
Thank you so much in advance!
"determine the normal closure" is not very clear. It seems you want to identify the isomorphism type of the normal closure.
Namely, this kernel $\mathrm{BS}_0(m,n)$ is an infinitely iterated amalgam $$\cdots\ast_\mathbf{Z}\mathbf{Z}\ast_\mathbf{Z}\mathbf{Z}\ast_\mathbf{Z}\mathbf{Z}\ast_\mathbf{Z}\cdots$$ where each left embedding is $k\mapsto nk$ and each right embedding is $k\mapsto mk$. This is a general fact about HNN extensions (description of the isomorphism type of the kernel of the canonical homomorphism from a given HNN-extension onto $\mathbf{Z}$), which can be found in Serre's book. In particular, this kernel $\mathrm{BS}_0(m,n)$ is not free as soon as $\max(|m|,|n|)\ge 2$. Indeed,
On the other hand, the kernel $\mathrm{BS}_{00}(m,n)$ of the canonical homomorphism onto $\mathbf{Z}[m/n,n/m]\rtimes_{m/n}\mathbf{Z}$ is free, since it acts freely on the Bass-Serre tree.
Note: $\mathrm{BS}_0(m,n)$ is locally residually finite. Also $\mathrm{BS}_0(m,n)$ is not residually finite if $|m|,|n|$ are coprime and $\ge 2$: this is due to R. Campbell (1990 Proc AMS). I don't know if it's also non-residually-finite whenever $2\le |m|<|n|$, for instance, $(m,n)=(2,4)$.
Edit: the free subgroup $\mathrm{BS}_{00}(m,n)$ is infinitely generated as soon as $2\le |m|<|n|$. Since $2\le \min(|m|,|n|)$, it is not trivial, hence contains a loxodromic element for the action on the Bass-Serre tree. Hence it has a unique minimal nonempty subtree (the convex hull of the union of axes of loxodromic elements) for the action on the Bass-Serre tree $T$ of the HNN extension defining $\mathrm{BS}(m,n)$; hence this subtree is invariant under the whole group, and hence by vertex-transitivity, this is the whole tree.
The tree $T$ naturally carries a Busemann-function (defined up to addition with an integral constant), so that every vertex $v$ has $n+m$ adjacent vertices: $m$ vertices $w$ with $b(w)=b(v)-1$ and $n$ vertices $w$ with $b(w)=b(v)+1$. Let $G$ is the group of automorphisms of $T$ preserving $b+\mathbf{Z}$, so the locally compact group $G$ acts cocompactly on $T$ and $\mathrm{BS}(m,n)$ acts through $G$.
Now assume by contradiction that $\mathrm{BS}_{00}(m,n)$ is finitely generated: then since it acts freely and minimally on the tree $T$, the action is cocompact. Hence $\mathrm{BS}_{00}(m,n)$ is a cocompact lattice in $G$. But $G$ is not unimodular, by an easy argument (using $|m|\neq |n|$). This is a contradiction.