I came across a seemingly standard but interesting statement:
Given a finite set of polynomials in $\mathbb{Z}[x_1,...,x_n]$, show that they have a common solution in $\mathbb{C}$ if and only if they have a common solution in a finite field of characteristic p for infinitely many values of p.
Firstly, I am looking for a proof of this statement.
Secondly, Is there a similar criterion for an infinite set of such polynomials?
Finally, I also am looking for examples where this criterion is 'useful' in that it might be difficult to see a common solution in $\mathbb{C}$ but easier to see solutions in infinitely many finite fields (and vice versa).
Edit: Thanks to the comments, the second question regarding infinitely many polynomials is clear. Since $\mathbb{Z}$ is Noetherian, any ideal is finitely generated and so it makes sense that finding common solutions to infinitely many polynomials is the same as finding the solutions to the finite set of polynomials that generate the same ideal.
It is probably possible to give a purely algebraic proof of this statement, but it also follows easily using standard facts from mathematical logic. So I hope you don't mind if I use this answer as an advertisement for the field of model theory.
First, note that a system of polynomial equations in $\mathbb{Z}[x_1,\dots,x_n]$ has a solution in a finite field of characteristic $p$ if and only if it has a solution in $\overline{\mathbb{F}_p}$, the algebraic closure of the prime field of characteristic $p$. Indeed, every finite field $K$ of characteristic $p$ embeds in $\overline{\mathbb{F}_p}$, and the image of a solution in $K$ under such an embedding is a solution in $\overline{\mathbb{F}_p}$. And conversely, given a solution $(a_1,\dots,a_n)\in (\overline{\mathbb{F}_p})^n$, this is also a solution in the finite subfield $\mathbb{F}_p[a_1,\dots,a_n]\subseteq \overline{\mathbb{F}_p}$.
So we have reduced to showing that a system of polynomial equations in $\mathbb{Z}[x_1,\dots,x_n]$ has a solution in $\mathbb{C}$ if and only if it has a solution in $\overline{\mathbb{F}_p}$ for infinitely many values of $p$. In fact, it turns out that this is also equivalent to having a solution in $\overline{\mathbb{F}_p}$ for all but finitely many values of $p$.
Now the reason this three-way equivalence is true is that given a system $\{p_1,\dots,p_k\}$ of polynomials with coefficients in $\mathbb{Z}$ (which we can assume to be finite, by Noetherianity of $\mathbb{Z}[x_1,\dots,x_n]$, as noted in the comments), we can express the existence of a solution by a sentence of first-order logic in the language of fields: $$\exists x_1\dots\exists x_n\,(p_1(\overline{x}) = 0\land \dots \land p_k(\overline{x}) = 0).$$ Now we can apply:
Theorem: Any sentence of first-order logic in the language of fields is true in $\mathbb{C}$ if and only if it is true in $\overline{\mathbb{F}_p}$ for infinitely many primes $p$ if and only if it is true in $\overline{\mathbb{F}_p}$ for all but finitely many primes $p$.
A closely related application of the Theorem is the Ax-Grothendieck theorem, the model-theoretic proof of which (due to Ax) lifts the fact that every injective function from a finite set to itself is surjective up to polynomial maps between algebraic varieties over $\mathbb{C}$.
The proof of the above Theorem is usually phrased using the compactness theorem. But here's another way to think of it, which may seem more intuitive if you're less familar with mathematical logic. By $\text{ACF}_0$, I mean the axioms of algebraically closed fields of characteristic $0$. These are the field axioms, together with an axiom for each degree $d$ saying that all non-constant polynomials of degree $d$ have roots, and finally an infinite list of axioms saying the characteristic is not any prime: $2\neq 0$, $3\neq 0$, $5\neq 0$, etc. Now it's a fact that $\text{ACF}_0$ is a complete theory, meaning that we can prove or disprove every sentence in the language of fields using the axioms of $\text{ACF}_0$. [This is a nontrivial fact, which is where all the algebra in the proof is hiding. You can also understand this as saying that if a sentence holds of one algebraically closed field of characteristic $0$, then it holds of all such fields, i.e. a manifestation of the Lefschetz principle.] In particular, if a sentence $\varphi$ is true in $\mathbb{C}$, then it must be provable from the axioms of $\text{ACF}_0$. But the proof of $\varphi$ will only use finitely many of the characteristic axioms (proofs being finite). So it follows that $\varphi$ is provable from the axioms of algebraically closed fields, together with finitely many axioms of the form $p\neq 0$ for finitely many primes $p$. Hence $\varphi$ is true in all algebraically closed fields except possibly for those whose characteristics are among these finitely many exceptional primes.
Of course, if $\varphi$ is true in the $\overline{\mathbb{F}_p}$ for all but finitely many primes $p$, then it is true in the $\overline{\mathbb{F}_p}$ for infinitely many primes $p$ (by Euclid's theorem). To complete the chain of equivalences, note that by the argument above, if $\varphi$ is true in infinitely many of the $\overline{\mathbb{F}_p}$, then it can't be that $\lnot \varphi$ is true in $\mathbb{C}$, otherwise we could run the argument in the last paragraph for the sentence $\lnot \varphi$, and find that $\lnot\varphi$ is true in all but finitely many of the $\overline{\mathbb{F}_p}$. Thus $\varphi$ must be true in $\mathbb{C}$.