Let $F$ be a sheaf of commutative rings or Abelian groups on a topological space $X$, let $x \in X$ be a point, let $U$ be an open neighborhood of $x$ in $X$. Let $f \in F(U)$, and suppose the germ $f_x$ of $f$ at $x$ is 0.
My question is, does it follow that there is some open $V$ with $x \in V \subset U$, and $f|_V = 0 \in F(V)$?
I think the answer is yes if the sheaf $F$ is valued in Abelian groups; this uses the explicit construction of the stalk as a colimit, and a section will only become 0 in this colimit if it can be expressed as a (finite) sum of sections (on possibly smaller neighborhoods) whose sum restricts to 0 on some common smaller neighborhood.
However, I'm not sure how a similar statement if $F$ is valued in commutative rings, since the explicit colimit construction in commutative rings seems more complicated.
Would anyone have any suggestions on how to think about this, or know a reference that describes this? Also, is my reasoning above correct?
Yes.
Suppose $\mathcal F$ is a sheaf of abelian groups.
The definition of $\mathcal F_x$ and the meaning of $0\in \mathcal F_x$.
The stalk of $\mathcal F$ at $x\in X$, denoted $\mathcal F_x$, is defined as the colimit $$\varinjlim_{U\ni x}{\mathcal F}(U)$$ where $U$ ranges over all the open sets containing $x$ ordered by reverse inclusion. In other words, an element of $\mathcal F_x$ is an equivalence class in $\displaystyle{\bigsqcup_{U\ni x}\mathcal F(U)}$, where two sections $f_{U_1}\in\mathcal F(U_1)$ and $g_{U_2}\in\mathcal F(U_2)$ are equivalent if the restrictions of the two sections coincide on some open neighborhood of $x$ that is a subset to both $U_1$ and $U_2$. Given the open neighborhood $U$ of $x$ and the section $f\in\mathcal F(U)$, $f|_x$ is the equivalence class in $\displaystyle{\bigsqcup_{U\ni x}\mathcal F(U)}$ that contains $f$.
For clarity, the zero element in an abelian group $A$ will be denoted $0_A$. The set $\mathcal F_x$ can be made into an abelian group naturally, where $0_{\mathcal F_x}$ is the equivalent class in $\displaystyle{\bigsqcup_{U\ni x}\mathcal F(U)}$ that contains $0_{\mathcal F(X)}$. You may want to verify the last statement in detail.
A proof
Suppose $f_x$, the germ of $f\in\mathcal F(U)$ at $x$ is $0$, i.e., $0_{\mathcal F_x}$. As described above, that means $f$ and $0_{\mathcal F(X)}$ are in the same equivalence class in $\displaystyle{\bigsqcup_{U\ni x}\mathcal F(U)}$, i.e., the restriction of $f$ and $0_{\mathcal F(X)}$ coincides on some open neighborhood of $x$. Denote that open neighborhood by $V$. Then $$\begin{align} f|_V &=(0_{\mathcal F(X)})|_V\tag{1a}\\ &=0_{\mathcal F(V)}\tag{1b}\\ &=0\in\mathcal F(V)\tag{1c} \end{align}$$ Explanations:
$(1a)$: the meaning of $f$ and $0_{\mathcal F(X)}$ coincides on $V$.
$(1b)$: Restriction morphism is compatible with the abelian groups.
$(1c)$: Just another notation
The explanations/proof above are valid verbatim if $\mathcal F$ is a sheaf of rings instead, once "abelian group" is replaced with "ring" everywhere.