This is probably a very silly question:
If $R$ is an arbitrary commutative ring with unit and $f\in R[X]$ a polynomial, then for any element $a\in R$ we have $$f(a)=0 \Longleftrightarrow X-a ~\mbox{ divides }~ f \Longleftrightarrow f\in (X-a)$$ where the last equivalence is clear. The first is probably a little surprising as $R[X]$ is usually not euclidean and it is perhaps not clear how to divide by $X-a$.
Now let $f\in R[X_1,\ldots, X_n]$ be a polynomial. How can I see for an element $(a_1,\ldots,a_n)\in R^n$ that $$f(a_1,\ldots,a_n)=0 \Longleftrightarrow f\in (X_1-a_1,\ldots,X_n-a_n) ?$$ If this does not work in general, let $R=K$ be a field.
Yes. Another way to see this is by the following:
We can assume without loss of generality that $X_n$ appears in $f$. Then $$f=\varphi_0+\varphi_1X_n+\cdots + \varphi_kX_n^k$$ for some $\varphi\in R[X_1,...,X_{n-1}]$ with $\varphi_j(a_1,..,a_{n-1})\neq 0$ for some $j\leq k$. This reduces to the case you are okay with!