For a prime ideal $p$ in Spec $A$, consider the $0$ element of the stalk $\mathcal{O}_p$, where $\mathcal{O}$ is the structure sheaf.
If $f$ is equivalent to the $0$ element of the stalk, then does it have to be $0$ in a neighborhood containing $p$, or does it have to be $0$ only at $p$? I thought that it has to be $0$ in a neighborhood, but reading a proof in Hartshorne is making me think otherwise.
Let $U$ be an open subset of ${\rm Spec}(A)$ containing the point $\mathfrak p$, and let $f$ be a section in $\mathcal O(U)$. For $f$ to vanish in the stalk $\mathcal O_{\mathfrak p}$, we need $f$ to vanish as an element of the localisation $A_{\mathfrak p}$. That's all the definition requires.
However, if it is the case that $f$ vanishes in the stalk $\mathcal O_{\mathfrak p}$, then it is easy to show that there must exist an open neighbourhood $V \subset U$ of $\mathfrak p$ such that $f$ vanish in the stalks $\mathcal O_{\mathfrak q}$ for all $\mathfrak q \in V$.
Some examples:
$A = k[x]$, $\mathfrak p = (x)$, $f = 0$. This $f$ vanishes in $\mathcal O_{\mathfrak p} = A_{\mathfrak p}$. But $\mathfrak p$ is not the only point where $f$ vanishes: $f$ also vanishes in $\mathcal O_{\mathfrak q}$ for all $\mathfrak q \in {\rm Spec}(A)$.
$A = k[x,y]/(xy)$, $\mathfrak p = (x - 1)$, $f=y$. (You can think of the space as the union of the $x$ and $y$ axes.) $f$ vanishes in the localisation $\mathcal O_{\mathfrak p} = A_{\mathfrak p}$, by virtue of the fact that $xf$ is zero in $A$, and $x \notin \mathfrak p$. But $\mathfrak p$ is not the only point where $f$ vanishes: $f$ also vanishes in $\mathcal O_{\mathfrak q}$ for all $\mathfrak q \in D(x) \subset {\rm Spec}(A)$, where $D(x)$ is the open subset of ${\rm Spec}(A)$ consisting of the prime ideals in $A$ that do not contain $x$.
For the general case, suppose $A$ is a ring, $\mathfrak p$ is a prime ideal, and $f = g/h$ is an element of $\mathcal O_{\mathfrak p} = A_{\mathfrak p}$ (with $g \in A$ and $h \in A \setminus \mathfrak p$). If $f$ vanishes as an element of $\mathcal O_{\mathfrak p} = A_{\mathfrak p}$, then there exists an element $e \in A \setminus \mathfrak p$ such that $eg$ vanishes as an element of $A$. So $f$ also vanishes in $\mathcal O_{\mathfrak q}$ for all $\mathfrak q$ in the open neighbourhood $V = D(e)\cap D(h) \subset {\rm Spec}(A)$. (In fact, $f$ also vanishes as a section in $\mathcal O(V)$.)
By the way, if you don't want to fiddle around with localisations, there is a purely sheaf-theoretic way of handling this. The stalk $\mathcal O_{\mathfrak p}$ is the set of equivalence classes $[(U, f)]$ (with $\mathfrak p \in U$), where $(U_1, f_1) \sim (U_2, f_2)$ iff there exists an open $V \subset U_1 \cap U_2$ containing $\mathfrak p$ such that $f_1|_{V} = f_2|_{V}$. The zero element of the stalk $\mathcal O_{\mathfrak p}$ is the equivalence class of $({\rm Spec}(A), 0)$. So if a section $f$ in $\mathcal O(U)$ restricts to the zero element in $\mathcal O_{\mathfrak p}$, then there must exist an open neighbourhood $V \subset U$ containing $\mathfrak p$ such that $0|_V = f|_V$. But then, $f$ restricts to the zero element in $\mathcal O(V)$, and so, $f$ restricts to the zero element in $\mathcal O_{\mathfrak q}$ for every $\mathfrak q \in V$.