Suppose any two non-trivial (non-singleton) subgroups of a group $G$ have a non-trivial intersection. $|G|$ is necessarily a prime power, because by Cauchy's theorem, for any prime $q$ dividing $|G|$, $G$ must contain an element, and therefore a subgroup of order $q$. So for any two distinct primes $q, q_2$ dividing $|G|$, the subgroups of orders $q$ and $q_2$ will have trivial intersection.
So suppose that $|G|=q^k$ for some prime $q$ and positive integer $k$. Is it possible to construct $G$ such that for all subgroups $H$ of $G$, $J \subset H$ or $J \subset H$, depending on if $|J|$ < $|H|$? It is straightforward to deduce the fact that $J$ must be a cyclic subgroup.
When $q=2$ ($k$ is arbitrary), the quaternion group is the smallest example of such a group (since $J={\{1,-1}\}$ is a subgroup of all non-singleton subgroups of the quaternion group.
Is there a $3$-group i.e. $|G|=3^k$ which has only two elements of order three? The non-abelian groups of 27 can all be written as semidirect products, so the smallest possible order is $81$.
What about when $J$ is a subgroup of order $4$, $J = {\{1, i, -1, -i}\}$, what is the smallest example of a group where every subgroup containing an element of order $> 4$ contains $J$ as a subgroup?
The idea here is similar to a semidirect product, but with $N \cap H=J$, where $N$ is a normal subgroup of $G$, and $H$ is a subgroup of $|G|$.
The semi-direct product generalizes the direct product towards group actions, while the central product generalizes the direct product towards cohomology. Note that $p$-groups are determined entirely by cohomology. The central product construction sounds close to what you're after.
More generally, sections 1-4 of this paper (https://epubs.siam.org/doi/10.1137/15M1009767) are likely to serve as good reading material.