The set $A=\{1, \frac{1}{2}, \frac{1}{4}, \dots \}$ is obviously not closed in $\mathbb{R}$ with the Euclidean metric, as the sequence $1, \frac{1}{2}, \frac{1}{4} \dots$ converges to $0 \notin A$.
But if we look at the complement, $A'$ and let $p\in A'$ then there exist some $a_i$ and $a_{i+1}$ such that $a_{i+1}<p<a_i$. Letting $\epsilon=\frac{a_i-a_{i+1}}{2}$ we see that the open ball $B_\epsilon(p) \cap A = \emptyset$ so $B\epsilon(p) \subset A'$ thus proving that $A'$ is open and $A$ closed.
What am I doing wrong?
Your complement $A'$ includes $0$ but you cannot find an open ball around it. Your mistake is tacitly assuming $p \in A'$ is in $(0,1)$ as well.