Let's consider the group extension $G$ s.t. $$1 \to A \to G \to B \to 1$$ where $A$ and $B$ are finite, non-abelian and simple groups. Let $C_G(A)$ be the centralizer of $A$. Since $A$ is normal, $C_G(A)$ is normal as well. Furthermore, the intersection of a subgroup of a group and its centralizer is just the center of the subgroup, and we know that non-abelian simple groups are centerless. So $A\cap C_G(A) = 1$ and $AC_G(A) = A\times C_G(A)$.
From here how can we conclude that $C_G(A)$ is isomorphic (as a group) to a normal subgroup of $B$? For one, $B$ might not be (isomorphic to) a subgroup of $G$! Second, I don't see why $C_G(A)$ should be normal in $B$. Moreover, $B$ might not even be a complement of $A$ unless $G$ is a split extension (cf. this).
I'm trying to understand the claim made in the second paragraph of this answer.
Consider the image of $AC_G(A)$ in $B$ under the map $G\to B$. The kernel of the map is $A$. So the image is isomorphic to $$\frac{AC_G(A)}{A} \cong \frac{C_G(A)}{C_{G}(A)\cap A} = \frac{C_G(A)}{\{e\}} \cong C_G(A).$$ Thus, there is a subgroup of $B$ that is isomorphic to $C_G(A)$.
Since the map $G\to B$ is surjective, it maps normal subgroups of $G$ to normal subgroups of $B$. Since $AC_G(A)$ is the product of two normal subgroups of $G$, it is normal in $G$. So its image is normal in $B$. But the image is isomorphic to $C_G(A)$. So $B$ contains a normal subgroup that is isomorphic to $C_G(A)$.
Since $B$ is assumed to be simple, this means $C_G(A)$ is either trivial or isomorphic to $B$. In the latter case, it turns out that $G$ contains a subgroup isomorphic to $B$ that intersects $A$ trivially, and by order considerations you get $G=A\times C_G(A)\cong A\times B$ and the extensions is split.
Note that the claim is not that $C_G(A)$ itself is a subgroup of $B$, but rather that it is isomorphic to a subgroup of $B$.