I'm trying to follow an argument made in one of Benedict Gross's online abstract algebra lectures.
He first proves that given $W \subset V$ a subspace of dimension $m$ (ambient vector space $V$ has dimension $n$), then it can be extended to a basis by adding vectors $v_{m+1}, \ldots, v_n$. The claim is then that if $f$ is the canonical homomorphism $V \to V/W$, then $f(v_{m+1}), \ldots, f(v_n)$ is a basis for $V/W$. (I haven't proved this, but for the moment I'm taking it for granted.) So $W' = \mathrm{span}(v_{m+1}, \ldots, v_n)$ is isomorphic to $V/W$, so $V/W$ can be thought of as a subgroup of $V$.
The key insight is that this cannot be done with groups, but I don't fully understand the details. First, we notice that \begin{align*} \mathbb{Z}/2\mathbb{Z} \cong \frac{2\mathbb{Z}}{4\mathbb{Z}} \leq \mathbb{Z}/4\mathbb{Z}, \end{align*} which is fine. Call $H = \frac{2\mathbb{Z}}{4\mathbb{Z}}$ and $G = \frac{\mathbb{Z}}{4\mathbb{Z}}$. The argument is that, then, unlike in the case of vector spaces, I can't find another cyclic group of order $2$, $H'$ s.t. $G/H' \cong G/H$ because $H \to 0$ in the homomorphism $f$.
I'm only understanding bits and pieces of this. I'm hoping someone can clarify this.
Every quotient vector space $V/W$ is isomorphic to a vector subspace of $V$, by the argument you cite.
Gross' point is that the analogous claim does not hold for groups: not every quotient group $G/H$ is isomorphic to a subgroup of $G$. But some quotient groups do admit such an isomorphism! Your example is one of them: you have $G/H\cong H\leq G$.
When there exists such an isomorphism, we say the quotient map "splits".
For an example that does not split, consider $G=\mathbb{Z}$ and $H=2\mathbb{Z}$. Then $G/H$ contains an element $x$ such that
Any group isomorphic to $G/H$ must contain such an element, so any group containing an isomorphic copy of $G/H$ must hold such. But $G$ contains no such element.