I want to start with the definition of a linear manifold I am using:
A subset $A\subset \mathbb{R}^n$ is called $k$-dimensional $\mathcal{C}^\alpha$-submanifold ($\alpha \ge 1$) if for every point $a\in M$ there is an open neighbourhood $U\subset \mathbb{R}^n$ and $n-k$ functions $f_j\in\mathcal{C}^\alpha\colon U\rightarrow \mathbb{R}$ such that
$M\cap U=\{x\in U \colon f_1(x)=...=f_{n-k}(x)=0\}\\ \text{rank}\frac{\partial (f_1,...,f_{n-k})}{\partial x_1,...,x_n)}(a)=n-k$
If I remove finitely many points from $M$, I can show that $M$ is still a submanifold, even having the same defining functions as above which are then restricted. (If a point $p$ is removed, and $p\in U$, just pick another (smaller) neighbourhood not cointaining $p$)
So removing finitely many points should not be a problem.
If I were to remove countably infinite points, I can not do the same and there I am not sure if it is still a manifold.