I believe I need to construct examples for both cases.
$\mathbb{Q}$ is a countable subset of $\mathbb{R}$ and $\mathbb{Q}$ is not closed in $\mathbb{R}$ since $\mathbb{R}-\mathbb{Q}$ is not open in $\mathbb{R}$.
The set of all primes $\mathbb{P}$ is a countable subset of $\mathbb{R}$ which is closed in $\mathbb{R}$.
Are my examples correct?
Yes, they are correct.
Another example:
$\left\{\frac1n : n \in \mathbb{N}\right\}$ is not closed in $\mathbb{R}$, but $\left\{\frac1n : n \in \mathbb{N}\right\} \cup \{0\}$ is closed in $\mathbb{R}$.