Jacobi's formula says that:
$$\det e^{X}=e^{\operatorname{Tr}(X)}$$
So for any matrix $A$, I could try to find a matrix $X$ (the equivalent to a group generator) such that $A=e^{X}$ holds. But if $\det A$ is negative or zero, then $\det A=\det e^{X}=e^{\operatorname{Tr}(X)}$ would not be true, because $e^{\operatorname{Tr}(X)}$ is always bigger than zero.
Where is the failure in my reasoning?
There's no failure, unless you count assuming that such a matrix $X$ exists for any $A$. You've in fact given a proof that if $\det{A}\leq0$, there is no $X$ (with real trace) such that $e^X=A$.
If you allow $\operatorname{Tr}(X)$ to be complex (so it is possible for $e^{\operatorname{Tr}(X)}$ to be negative), then there does exist $X$ with $e^X=A$ for any invertible $A$, but not for general $A$. This $X$ is not generally unique.