Let $A \subseteq B$ be an integral extension of rings. Let $b \in B$, and let $b^n+a_1b^{n-1}+\dots+a_{n-1}b+a_n=0$ be an integral dependence relation for $b$, with $a_1,\dots,a_n\in A$.
Given a prime ideal $\mathfrak{p}$ in $B$, I am trying to prove that $b\in\mathfrak{p}$ if and only if $a_1,\dots,a_n\in \mathfrak{p}\cap A$. I have managed to prove the implication $\Leftarrow$, but I am stuck with the other: I am only able to show the claim for $a_n$.
In particular, I am interested in the case where $A$ is the subring of invariants under some group action on $B$, but since I proved one implication in this more general setting I suspect that the other direction may hold too under these assumptions.
It is not true in general. Note that $b\in\mathfrak{p}\Leftrightarrow a_1,\dots,a_n\in\mathfrak{p}\cap A$ implies $$f\left(D(b)\right)=\bigcup_i D(a_i)$$ where $f:\operatorname{Spec} B\to\operatorname {Spec} A,\mathfrak{p}\mapsto A\cap\mathfrak{p}$ is the induced map on the spectra. In particular $f$ would be an open map, and hence going-down would hold for the extension $B/A$. But there are integral extensions for which going-down does not hold (see e.g. here or here).
I don't know about the general case for the invariant ring under group actions but the following special case is true: If $G$ is a finite group that acts on $B$ and $A=B^G$ is the ring of invariants, then for $b\in B$ the claim (almost, see edit) holds for the polynomial $$\prod_{\sigma\in G}(X-\sigma(b))\in A[X]$$ (Since the coefficients of that polynomials are sums of products of elements of the form $\sigma(b)$)
Edit: It is also not true for the invariant ring, here is an easy example (much easier than the argument above): Let $B=\Bbb Z[i]$ be the ring of integers in $\Bbb Q(i)$ and $A=\Bbb Z$. The element $b=i+2$ satisfies $b^2-4b+5=0$ and we have $b\in \langle b\rangle =:\mathfrak{p}$, but $4\notin \mathfrak{p}\cap A=\langle 5\rangle$.
Note that $A$ is the ring of invariants under the Galois action on $B$.
What is true is the following (in the general setting again): If $b$ is contained in all prime ideals $\mathfrak{p}$ lying over the same prime $\mathfrak{q}$ of $A$ then also all $a_i$ lie in $\mathfrak{q}$. Here the $a_i$ are defined by $$\prod_{\sigma\in G}(X-\sigma(b))=X^n+a_1X^{n-1}+\dots+a_n$$ Since the elements $\sigma\in G$ permute the primes lying over $\mathfrak{q}$ all sums of products of elements of the form $\sigma(b)$ are also contained in all primes $\mathfrak{p}$ lying over $\mathfrak{q}$. Now the coefficients $a_i$ are exactly of this form and are furthermore contained in $A$, so we have $a_i\in \mathfrak{p}\cap A=\mathfrak{q}$