I tried reading the Wikipedia page but it's stated in terms of complex roots, and I don't really understand how that relates to the following proposition:
if a real valued polynomial:
$$\sum_{i=0}^n a_i x^i = 0$$
for all $x \in \mathbb{R}$, is it right to say the Fundamental Theorem of algebra implies that $a_i = 0$ for all $i$?
This was stated off handedly during a recent talk and I am not sure if I heard it correctly, since when I look up the theorem it is difficult for me to understand the way that page is written
A bit pedantically, it really depends on what you take to be the "Fundamental Theorem of Algebra (FTA)".
Strictly speaking, the result classically called the FTA is an existence result. It simply says that every (non-constant) polynomial over $\mathbb{C}$ has a root. Continuing inductively, an easy corollary is that a degree $n$ polynomial has at least $n$ roots.
This is sometimes combined with a distinct result, i.e., the factor theorem, to conclude that a non-zero degree $n$ polynomial over $\mathbb{C}$ cannot have more than $n$ roots, and hence has exactly $n$ roots. Overtime, this slightly strengthened corollary has also become known as the "Fundamental Theorem of Algebra".
But note that the fact that a non-zero degree $n$ polynomial over $\mathbb{C}$ cannot have more than $n$ roots technically has nothing to do with the FTA, which is a statement about the algebraic closure of $\mathbb{C}$. In fact, the upper bound on the number of roots holds for rings much more general than $\mathbb{C}$, and has a much easier proof than the FTA.
Theorem: Let $R$ be an integral domain. Then any non-zero degree $n$ polynomial over $R$ has at most $n$ roots counting multiplicity.
For a proof, see here.
The result you want would follow from applying the above theorem for $\mathbb{R}$, for example.