I know that this is addressed in a corollary of Langrange's theorem, which states that if a group $H$ has a prime order, there exists no nontrivial subgroups, since the cosets of any subgroup must partition $H$ and the cosets are the same size. But which group property is violated in $H$ if there does exist a proper subgroup $G$?
For example, if the group $G=(\{1,3,5,7\},·)$ where · is multiplication mod 8, the semigroup $H=G\cup \{2,4,6\}$ is not a group because it isn't closed ($2·4\notin H$). Is it closure which is always violated, or does it differ from case to case?
$H=(\mathbb Z_3,\times)$ is a semigroup of size $3$. It has a subgroup $G$ of size $2$. Try the argument from groups to see why it fails here.
The argument falls apart because the cosets are not disjoint, because $ab=ac$ does not mean $b=c$. So it is the existence of an inverse, and associativity, that allows us to make cosets partition $H$.