When is a locally Borel set Borel?

Question

Let $X$ be a LCH space. Call $E\subseteq X$ locally Borel if $E\cap K$ is Borel for all compact $K\subseteq X$. Evidently, locally Borel sets form a $\sigma$-algebra and every Borel set is locally Borel.

My question is the following. Suppose $X$ is a group, or more generally there is a Radon measure $\mu$ on $X$ (using, say, the definition in Folland: finite on compact sets, outer regular on all Borel sets, and inner regular on all open sets) such that $\mathrm{supp}\,\mu=X$. Does it follow that every locally Borel set is Borel?

Note some assumption on $X$ like this is necessary: for example, the first uncountable ordinal has subsets which are locally Borel but not Borel. On the other hand, if $X$ is $\sigma$-compact, a set is Borel if and only if it is locally Borel.

Answer 1

Following a now-deleted comment which was on the right track, I'm now pretty certain the following groups are counterexamples:

1. $\mathbb{R}$ times an uncountable discrete group,
2. An uncountable restricted direct product of copies of $\mathbb{Q}_p$.

In particular, they each decompose as an uncountable coproduct (i.e. topological disjoint union) of pieces $X_i$ with Borel subsets $E_i$ such that the rank of the sets $E_i$ in the Borel hierarchy grows without countable bound. The union of the sets $E_i$ is then locally Borel but not Borel.

It still may be interesting to figure out what the "correct" assumptions on $X$ are, but I'll consider this particular question answered unless I am wrong in calling these counterexamples.