Suppose $G$ is a group, and there exists a finite normal subgroup $H$ of $G$, such that $\frac{G}{H}$ is locally cyclic. Is it always true, that in this case $G$ has a locally cyclic subgroup of finite index?
I know how to solve this problem for the specific case, where $\frac{G}{H}$ is cyclic. If it is finite, then the whole group is finite and thus all its subgroups, even the cyclic ones, have finite indices. If $\frac{G}{H} = \langle a \rangle_\infty$, subbose $b$ is some element from the preimage of $a$. $\langle b \rangle$ is infinite cyclic and all cosets of $H$ are of the form $Hb^n$ for some $n$. Thus $G = H \rtimes \langle b \rangle$. ant thus $\langle b \rangle$ is cyclic of finite index.
However, I have no idea how to extend this to locally cyclic groups.
Yes. Let $A\le G$ be the centralizer of $H$; so $A$ has finite index in $G$ and $A/(A\cap H)$ is locally cyclic. So $A$ is central-by-(locally cyclic). Since central-by-cyclic implies abelian, it follows that central-by-(locally cyclic) implies abelian. Hence $A$ is abelian.
Now let's do the abelian case: $A$ is abelian, $F$ a finite subgroup with $A/F$ locally cyclic.
First case: $A/F$ is torsion; so $A$ is torsion. For every prime $p$ such that $F$ has no element of order $p$, the $p$-component $A_p$ of $A$ is locally cyclic, so we can can concentrate on $B=\prod_p A_p\le A$ where $p$ ranges over the finite set of primes $p$ dividing $|F|$. Then $B$ is finite and $B/F\simeq\mathbf{Z}[1/|F|]/\mathbf{Z}$. From the structure of artinian abelian groups, it follows that the divisible subgroup of $B$ has finite index in $B$ and is isomorphic to $\mathbf{Z}[1/|F|]/\mathbf{Z}$.
Otherwise $A/F$ is torsion-free. In this case $A\simeq (A/F)\times F$ by a classical theorem of Baer (see for instance this MO answer). But let me give a direct proof in this case to make this answer self-contained. I claim that there exists $n\ge 1$ such that $nA\cap F=\{0\}$. Indeed otherwise there exists a nontrivial element in $F\cap\bigcap_{n\ge 1}nA$, and hence $A$ has elements of arbitrary large finite order. But $A$ being finite-by-(torsion-free), this is a contradiction. Next, since $B=A/F$ is locally cyclic, $B/nB$ is finite for all $n\ge 1$. Hence $A/nA$ is finite for all $n\ge 1$. Then choose $n$ such that $nA\cap F=\{0\}$: then $nA$ has finite index and is isomorphic to a subgroup of $A/F$, so is torsion-free.