I think yes. Here is my approach.
As we know manifolds are locally Euclidean, Suppose we remove a point say $p$ from the manifold M then $p$ has a neighbourhood say U which is homeomorphic to an open subset V of $R^{n}$, let $\phi$ be the homeomorphism between them. Then, Consider any continuous map,
$ f: V-\phi(p) \rightarrow \{0,1\}$ where {0,1} has the discrete topology.
Then, it suffices to show that the map $f$ is constant.
Suppose $f$ is not constant then $\phi^{-1}(0)$ and $ \phi^{-1}(1)$ are not empty open sets in $V-\phi(p)$ more precisely $V-\phi(p)=\phi^{-1}(0) \cup \phi^{-1}(1)$. So, $\phi^{-1}(0)$ is a non trivial clopen set. Hence a contradiction.
And f is constant.
Now, $U$ is connected. I don't know how's to proceed and extend it to the whole manifold?
Thanks for any help!
Hint: Things get much easier if you use that a manifold is connected if and only if it is path connected.