It is well-known that a finite $p$-group with a unique subgroup of order $p$ is cyclic or generalized quaternion $Q_n$.
$$Q_n:= \langle x,y \mid x^{2^n} = y^4 = 1, x^{2^{n-1}} = y^2, y^{-1}xy = x^{-1} \rangle$$
The converse reduces to prove that $y^2$ is the unique element of order $2$ of $Q_n$.
Question: Is there a reference in which this converse is proved explicitly?
I know you asked for a reference, but this is an easy exercise, almost as short as a reference: $\langle x\rangle$ is a cyclic subgroup of index $2$, so it has a unique involution, and every element in the other coset has the same square: $(x^iy)^2=x^iyx^iy=x^iy^2y^{-1}x^iy=x^iy^2x^{-i}=y^2$. In particular, every element in the coset has order $4$.