This is Problem no. 37 from 17th chapter from Gallian's Abstract Algebra text.
[Gallian Chapter 17 Problem 37][1]
Is it necessary that $p(x)$ should be irreducible over $F$. Obviously, $F[x]/\langle p(x) \rangle$ would not be a field. But will the set $\{a + \langle p(x) \rangle | a \in F\}$ be a subfield of the ring $F[x]/ \langle p(x) \rangle$.
The arguments I used to prove this problem is similar to argument Fraleigh used while proving Kronecker's Theorem, and the fact that $p(x)$ is irreducible was never used except to show that $F[x]/\langle p(x) \rangle$ is a field. So I think the result is true for any non-constant polynomial.
If I am wrong please give an counterexample and part where my arguments use the fact that $p(x)$ is irreducible. This is the part of the proof from Fraleigh's text. [Fraleigh's proof of Kronecker's Theorem][2]