I haven't the slightest idea why (inner or outer) semi-direct group products are an interesting construction. I understand inner direct products, because you're just giving conditions for which a group can be considered the direct product of two of its subgroups, and I "get" direct products. They're a very simple construction, and showing that a group decomposes into that structure is a very strong statement.
But the outer semi-direct product construction seems totally arbitrary and bizarre. What's the intuition, here?
A very important question in group theory is the extension problem: given groups $H,N$, can you classify the groups $G$ that have $N$ as a normal subgroup with quotient $H$? In other words, groups that fit in a exact sequence $1 \to N \to G \to H \to 1$. Motivation from that comes from the classification of finite simple groups and the existence of composition series: if we could do all of that, and given these two other tools, we could classify all finite groups. That's very interesting!
As it turns out, the extension problem is very hard in general. The direct product $H \times N$ gives a "trivial" example of an extension of $H$ by $N$, but there's not a lot that can be said -- that's only one possible structure, there are many others.
There is one simplification though: split extensions. These are extensions such that there exists a section of the projection $p : G \to H$ (that is, a morphism $s : H \to G$ such that $ps = id$). This class of extension is exactly the class of semidirect products $N \rtimes H$. So semidirect products give a partial answer to the question of classifying group extensions.
I would also argue that the construction is not that far-fetched. I feel like the dihedral group is a great example. It's the set of isometries of a regular $n$-polygon. You've got rotations ($\mathbb{Z/nZ}$) and a symmetry with respect to a given line. Any element can be written as a product of a rotation and either the identity or the symmetry. But when you multiply two such elements, there's an interaction between the rotation and the symmetry and the rotation (it gets changed to the opposite rotation). If you think about it, it means that the dihedral group can be represented as a semidirect product $\mathbb{Z/nZ} \rtimes \mathbb{Z/2Z}$. So another possible answer is that the semidirect product is useful to construct interesting groups, for example the dihedral group.