My background is in computer engineering, not pure math. I've been studying the application of Finite Fields. Wikipedia on Finite field arithmetic says:
GF(p), where p is a prime number, is simply the ring of integers modulo p.
...
Elements of GF($p^n$) may be represented as polynomials of degree strictly less than n over GF(p).
I would really appreciate a simple, accessible explanation as to why this is the case? What fundamental property of Finite Fields makes it that the elements of $GF(p)$ and $GF(p^n)$ behave so differently and yet are both considered Finite Fields?
The field $GF(p)$, $p$ prime, is a so-called prime field and given by the integers modulo $p$.
The field $GF(p^n)$ is a field extension of $GF(p)$; i.e., it contains $GF(p)$ as subfield.
The practical way to construct $GF(p^n)$ is to choose a polynomial $f(x)$ in $GF(p)[x]$ of degree $n$, which is irreducible over $GF(p)$. Then the field $GF(p^n)$ is given by the quotient ring $GF(p)[x]$ modulo the ideal $\langle f(x)\rangle$; practically, the computations are modulo the polynomial $f(x)$.
There are other ways to consider $GF(p^n)$, e.g. as the zero-set of a polynomial - but this would be more abstract and not suitable for practical computations.