Let $A$ be the ring $\mathbb{Z}/60$. What does it look like under the localization at the prime (2)? Everything that is not divisible by 2 should be a unit, so one might thing this looks like $\mathbb{Z}/4$ - but on the other hand, rings always inject into their localizations...
I'm asking because Vakil in his notes says that the stalk at (2) of this ring is $\mathbb{Z}/4$. But this doesn't make sense to me because of the above confusion.
This is false: you've been spoiled by the case of integral domains! Actually this can even fail for integral domains: $ 0^{-1} \mathbb{Z}$ is the zero ring, since this forces $1 = 0 \cdot 0^{-1} = 0$.
In the more general case, localizations have the effect of discarding any "components" of your ring that are away from your ideal (this sentence makes more sense in the geometric picture). In this case, the components are $\mathbb{Z} / 4$, $\mathbb{Z} / 3$, and $\mathbb{Z} / 5$. So, localization at $(2)$ has the effect of focusing only on the $\mathbb{Z} / 4$ case, and discarding the other two components.
If you understand the example above of inverting zero, this might be more clear if you write it as $$ \left(\mathbb{Z}/60 \mathbb{Z}\right)_{(2)} = \{3^{-1}, 5^{-1}\} \mathbb{Z} / 60 \mathbb{Z} $$