$O(n)$ has two connected components, $(\det)^{-1}(1)$ and $(\det)^{-1}(-1)$. While I know it is not true, I am wondering why the above is not enough to say that $O(n)$ is the double cover of $SO(n)$ since $SO(n)$ can be identified with $(\det)^{-1}(-1)$.
Is this because there is no way to associate $(\det)^{-1}(1)$ to elements of $SO(n)$?
$\mathrm{O}(n)$ is a double cover of $\mathrm{SO}(n)$ in the purely topological sense. For any $R\in\mathrm{O}(n)\setminus\mathrm{SO}(n)$,
$$ \pi(A)=\begin{cases} A & A\in\mathrm{SO}(n) \\ AR & A\in\mathrm{O}(n)\setminus\mathrm{SO}(n) \end{cases} $$
is a double covering $\mathrm{O}(n)\to\mathrm{SO}(n)$. However, notice this is not a group homomorphism.
In the sense of group theory and Lie theory, a double cover implies the covering map is a homomorphism. In this context, $\mathrm{O}(n)$ is only a double cover of $\mathrm{SO}(n)$ in odd dimension.
Assuming a double covering homomorphism does exist, say its kernel was $\{I,R\}$ for $R\in\mathrm{O}(n)$. Since kernels are normal subgroups, this would force $R$ to be fixed by conjugation, i.e. central. The center is given by $Z(\mathrm{O}(n))=\{\pm I_n\}$, so the quotient is necessarily the projective orthogonal group $\mathrm{PO}(n)=\mathrm{O}(n)/\{\pm I_n\}$. If $n$ is even then $-I_n\in\mathrm{SO}(n)$ is in the identity component, so the quotient still has two components, unlike $\mathrm{SO}(n)$, therefore $\mathrm{PO}(n)\not\cong\mathrm{SO}(n)$ when $n$ is even. But when $n$ is odd, $\mathrm{O}(n)=\mathrm{SO}(n)\times\{\pm I_n\}$ is an internal direct product, so the quotient is $\mathrm{SO}(n)$.
Of course, even in odd dimension, this is a disconnected covering. When we speak of double covers of rotation groups, generally it's in the context of spin (look up "orientation entanglement" or "Dirac trick") and spinors (projective representations of $\mathrm{SO}(n)$), and this means considering the connected double cover $\mathrm{Spin}(n)$ (defined in general as a subset of the Clifford algebra $C\ell(n)$, but sometimes given equivalent classical matrix group representations in low dimension; look up "accidental isomorphisms"). We can also add multiple dimensions of space and time and instead talk about a symmetric bilinear form of signature $(p,q)$, in which case we have $\mathrm{Pin}(p,q)$ as a double cover of $\mathrm{O}(p,q)$ which maps component to component and restricts to a double cover $\mathrm{Spin}(p,q)$ of $\mathrm{SO}(p,q)$.
Compare with, for example, the double cover $S^1\to S^1$ given by square $z\mapsto z^2$ in the complex plane, vs. $S^1\times S^0\to S^1$ given by deleting the second component. The first is a connected double cover (imagine a twice-wound rubber band) while the second is a disconnected double cover (imagine two rubber bands on top of each other).