I've managed to show that for locally compact abelian groups, the connected component at the identity $G_0$ is equal to the intersection of all open subgroups of G, since we have that $$A(G_0) = \bigcup_{H \le G \\ H \ open}A(H)$$ where A(H) is the annihilator: Indeed, if $\chi$ is trivial on $G_0$, it induces a character $\psi$ on $G/G_0$ given by $\psi(gG_0) = \chi(g)$. But since $G/G_0$ is locally profinite and $\mathbb{S}^1$ has no small subgroups, there will exist a compact open subgroup $X$ of $G/G_0$ such that $\psi(X) = \{1\}$. If $\pi$ is the canonical projection on $G/G_0$, then $H = \pi^{-1}(X)$ will be an open subgroup of G and for any $h \in H$, $\chi(h) = \psi(\pi(h)) = 1$. Therefore, $\chi$ is in the union $\bigcup_{H \le G \\ H \ open}A(H)$. The other inclusion is trivial since any open subgroup of G will contain the connected component.
So this characterization of $A(G_0)$ implies $$G_0 = A(G_0)^{\bot} = \bigcap_{H \le G \\ H open} A(H)^{\bot} = \bigcap_{H \le G \\ H open} H$$ with the notation $X^{\bot} = \{g \in G: \chi(g) = 1 \ \forall \chi \in X\}$.
But it would make sense for this type of equality to remain true atleast for locally compact groups in general (considering the intersection over normal open subgroups), since one wouldn't expect an algebraic property like being abelian to have any influence over something topological, like the connected component at the identity.
So under what conditions does this equality still hold?
I guess it suffices to consider only hereditarily disconnected (that is when all components are one-point) topological groups, because by [Pon, $\S$23], if $G$ is a Hausdorff topological group and $N$ is the component of the identity of $G$ then the quotient group $G/N$ is hereditarily disconnected. Note first that we cannot drop the local compactness condition, because the claim fails for the additive group of rational numbers endowed with the usual topology. Each neighborhood of a hereditarily disconnected locally compact topological group $G$ contains an open subgroup $H$, see [Pon, Theorem 16]. Moreover, if the group $G$ is compact, then the subgroup $H$ can be chosen to be normal, see [Pon, Theorem 17]. On the other hand, according to [MZ], there exists a hereditarily disconnected locally compact group without normal compact open subgroups:
References
[MZ] D. Montgomery, L. Zippin, Topological transformation groups, Interscience Tracts in Pure and Applied Mathematics, vol. I, Interscience Publishers, Inc, New York-London, 1955.
[Pon] L.S. Pontrjagin, Continuous groups, 2nd ed., M., 1954, in Russian.