In my textbook it says a sequence $\{x_i\}$ converges to $x\in M$ if for every $\epsilon>0$ there exists a positive integer $N_\epsilon$ such that $D(x_n,x)<\epsilon$ for all $n>N_\epsilon$.
Suppose we have a metric space $M=(0,1]$. Is it then wrong to say that the sequence $\{x_n\}=\{1,\frac12, \frac13,\frac14,...\}$ is convergent, since $0\not\in M$?
I mean, as $n\rightarrow \infty$, $x_n$ gets arbitrarily close to $0$ and I'd say that means it converges to $0$. On the other hand, the definition seems to require that the limit is contained within $M$ in order to say it converges.
The sequence converges to $0$ in the reals, $0 \notin M$, so $D(0, \frac{1}{n})$ is not even defined! So the sequence does not converge in $M$.
A metric is a function $D: M \times M \rightarrow \mathbb{R}$, so distance is only defined for pairs where both are in $M$.
But it can be the case that in a larger metric space $(X,d)$ we have a subset $M$ that gets the restricted metric $d_M=d|_{(M \times M)}$, as in your case.
Then $x_n \rightarrow x$ in $(X,d)$ implies $(x_n) \rightarrow x$ in $(M ,d_M)$ iff $\forall n : x_n \in M \land x \in M$. If then however $x \notin M$, we have found that $M$ is not closed in $X$: we can "walk out of" $M$ with a sequence; $x$ is then a point not in $M$ that has points in $M$ lying as close as we like to it. Hence $x \in \overline{M}\setminus M$.