Let $G$ be a group and $H\le G$. The normal core of $H$ in $G$ is the kernel of the (left transitive) action of $G$ by left multiplication on the set $X$ of the left cosets of $H$, namely $\mathrm{Core}_G(H)=\bigcap_{a\in G}aHa^{-1}$. Similarly, if we take into account also the set of the right cosets of $H$, say $Y$, we can build up the (left) action $G\times (X\times Y)\to X\times Y$ defined by $g\cdot (aH,Hb):=(gaH,Hbg^{-1})$. Unless I've messed up anything, the kernel of this action ought to be $K:=\bigcap_{a,b\in G}(aHa^{-1}\cap b^{-1}Hb)=\mathrm{Core}_G(H)\cap\biggl(\bigcap_{a,b\in G, b\ne a^{-1}}(aHa^{-1}\cap b^{-1}Hb)\biggr)$, which would then be a subgroup of $\mathrm{Core}_G(H)$ normal in $G$.
So as to googling for it, in case, does the subgroup $K$ have a name?
It is just the core of $H$ (which is the largest normal subgroup of $G$ contained in $H$). You did not make it any smaller.
Note that each $b^{-1}Hb$ is of the form $aHa^{-1}$ (by taking $a=b^{-1}$), and vice-versa. So your intersection is just the intersection of all conjugates of $H$. Any element of $\mathrm{Core}_G(H)$ lies in both $aHa^{-1}$ and $b^{-1}Hb$ for all $a,b\in G$.