This is a sequel to Why can the homomorphism $\phi$ in semi-direct products only be varied by inner automorphisms upon changing the complement group?
My professor made another remark that:
Let's go back to the extension question
$1 \to A \to G \to B \to 1$
It turns out that a general extension of $B$ by $A$ is a mutual generalization of two extremes. One extreme is a semidirect product, in which $A$ is complemented but conjugation by $B$ (and therefore by elements of $G$ in general) can be highly non-trivial. In the other extreme, $A$ is a subgroup of the center $Z(G)$, so that conjugation in $G$ does nothing whatsoever to $A$; but the search for a complement of $A$ may be an utter failure. This extreme is called a central extension, and it is another construction that you should learn in the context of both Jordan-Holder and p-groups.
Questions:
In case of central extensions, why exactly can the search for a complement of $A$ be an utter failure?
What is meant when he says that conjugation by $B$ can be highly non-trivial in case of semi-direct products?
I believe that this definition of "central extension", as one with $A \le Z(G)$ in which there is no complementis non-standard. The standard meaning of "central extension" is simply one in which $A \le Z(G)$, irrespective of whether there are complements are not.
But let me give you three examples in which $A \le Z(G)$. In the first of these there is a complement, and in the second and third there is not.
Example 1. Let $G = \langle x \rangle$ be cyclic of order 6: so $x^6=1$ (the identity). Let $A = \langle x^3 \rangle$ be the subgroup of $G$ of order $2$, and $B \cong G/A$ is cyclic of order $3$. So $A \le Z(G)$: in fact $G = Z(G)$. Now let $C$ be the subgroup $\langle x^2 \rangle$ of $G$, which is cyclic of order 3 with $C \cong G/A$. Then $B$ is a complement to $A$ in $G$: in fact it is the unique such complement.
Example 2. Now let $G = \langle x \rangle$ be cyclic of order 4: so $x^4=1$. Let $A = \langle x^2 \rangle$ be the subgroup of $G$ of order $2$, and $B \cong G/A$ is cyclic of order $2$. Again we have $A \le Z(G)$ and $G = Z(G)$. This time there is no complement $C$ of $A$ in $G$. Such a complement would have to be isomorphic to $B$ (i.e. cyclic of order 2) and satisfy $A \cap C = \{1\}$. But $A$ is the only subgroup of $G$ of order $2$, so no such $C$ exists.
Example 3. Let $G = \langle 1,r,r^2,r^3,s,sr,sr^2,sr^3 \}$ be the dihedral group order $8$. That is the group of rotations and reflections of a square, where $r,r^2,r^3$ are rotations, and $s,sr,sr^2,sr^3$ are reflections. Let $A=Z(G)$, which is the subgroup $\langle r^2 \rangle$ of order $2$. Note that $B \cong G/A$ is a Klein four group. A complement $C$ of $A$ in $G$ would be a subgroup of order $4$ isomorphic to $B$ with $A \cap C = \{1\}$. There is no such subgroup. There are subgroups such as $\{1,r^2,s,r^2\}$ (in fact there are two such subgroups) that are isomorphic to $B$, but they all contain $r^2$.
As an exercise, try to think of an example in which $A = Z(G)$ and there are complements of $A$ in $G$.
I am afraid that I do not really know the answer to your second question. It seems to me like an informal comment, and I would advise you not to worry about it.