The characterisation of a matrix Lie group is the following: Suppose $(X_n)$ is a sequence of invertible matrices converging to some $X$ in a set $G$ and the determinant of $X$ is nonzero, then $X$ is in $G$.
In the text I am currently reading on the subject, the author claims that this condition of being closed under nonsingular limits is equivalent to the smooth structure of a manifold.
I can't see this at all.
This is a non-trivial result in Lie theory. It follows from the following more general result:
For a proof, see for example Lee Introduction to smooth manifolds, Theorem 20.12. Or also the Wikipedia page of that theorem.
The non-singular limit condition is precisely the statement that $G$ is a closed subgroup of $\mathrm{GL}(n,\mathbb{R})$ and hence a Lie group by the above theorem. Note that $\mathrm{GL}(n,\mathbb{R})$ is a Lie group since it is an open subset of the real vector space $\mathrm{Mat}(n,\mathbb{R})$ of all $n\times n$ matrices.