Say I want to classify all groups of a given order. The abelian case is completely understood by the structure theorem for finitely generated abelian groups.
Assume our group is non-abelian, and we somehow managed to find a normal subgroup $N$ (e.g., by considering the core of some Sylow subgroup). It would be pretty nice if we could go on to construct all semidirect products of $N$ with groups of the remaining order – but $G$ need not be semidirectly reducible*.
The only counterexample that comes to mind is $0 \to C_2 \to C_4 \to C_2 \to 0$, which does not admit an appropriate section, i.e. a subgroup complementing our normal subgroup with trivial intersection.
What are counterexamples in the non-abelian case?
What about $\mathrm{Aut}(A_6)$, where $A_6$ is the alternating group on six letters? $A_6$ is normal in $\mathrm{Aut}(A_6)$ and it has no complement. Or consider the Mathieu group $M_{10}$. It contains $A_6$ as a normal subgroup of index $2$ (indeed, $M_{10}^{'} = A_6$), but $A_6$ again has no complement in $M_{10}$. Although these groups are not simple, they are both almost simple.
If, instead, you are interested in soluble groups only, take an odd prime $p$ and consider the non-abelian group $P$ of order $p^3$ and exponent $p$. The centre of that group is $\cong C_p$ and it has no complement in $P$, so $P$ is a non-split extension of $C_p$ by $C_p \times C_p$. More generally, if $G$ is a non-abelian group and $G/Z(G)$ is abelian (we can guarantee this, for example, by requiring $|G:Z(G)| = p^2$ for some prime $p$) then $Z(G)$ can have no complement in $G$. (Do you see why?)
Of course, if we can find a normal subgroup $N$ of $G$ and $\gcd(|N|,|G:N|)=1$, then $G$ is a semi-direct product, because $N$ is guaranteed to have a complement in $G$ by the Schur-Zassenhaus theorem.