I observed that all examples of localizations of polynomial rings at prime ideals I've encountered have been, so far, valuation rings, and so I started wondering whether this is true in all cases. I think the following might be an argument.
We have the field of fractions of the localized polynomial ring: $$\text{Frac}(R[x_1,\dots,x_n]_{\mathfrak p}) = \{ \frac{f_1s_2}{f_2s_1}: f_1,f_2,s_1,s_2 \in R[x_1,\dots,x_n]; s_1, s_2 \notin \mathfrak p \}$$ Consider an arbitrary element of $R(x_1, \dots, x_n)$. Then the numerator is either in $\mathfrak p$ or not. If it is, then $\text{num} = f \cdot 1$. If it isn't, then $\text{num} = 1 \cdot s$. The same argument can be done for denominator, and therefore we have $\text{Frac}(R[x_1,\dots,x_n]_{\mathfrak p}) = R(x_1 \dots, x_n)$, from which it clearly follows $f \in R(x_1, \dots, x_n)$ implies $f^{-1}$ or $f \in R[x_1,\dots,x_n]_{\mathfrak p}$, since either the denominator is in $\mathfrak p$ or it is not.
For a field $F$, even $R=F[x,y]$ does not have the property that all its localizations at primes are valuation rings.
There are a lot of ways to see it, but try this one: $R$ has Krull dimension $2$, and so will its localization $S$ at $(x,y)$. $S$ is already a Noetherian domain, and if it were a valuation ring it would have to be a PID, hence have Krull dimension less or equal to $1$. So $(x,y)$ is a prime (maximal, even) ideal of $R$, for which the localization is not a valuation ring.