I was reading Dummit and Foote and the following remark showed up:
Note that it is not true that if $I$ is a maximal ideal of $R$ then $I[x]$ is a maximal ideal of $R[x]$.
If $I$ is maximal in $R$ then $R/I$ is a field and this means that $R[x]/I[x]$ is a field since $R[x]/I[x]\cong (R/I)[x].$ Beyond this, I am not sure how I can come up with an example of this claim.
Let $R = \mathbb Z$. Then $I = 2\mathbb Z $ is a maximal ideal in $R$.
Now $I[x] = (2)$ is the ideal of polynomials with even coefficients in $R[x] = \mathbb Z[x]$. This is not a maximal ideal, since $(2,x)$ is a proper ideal of $\mathbb Z[x]$ that strictly contains $I[x]$.
As for the argument in your final paragraph, it is not true that $F$ being a field implies that $F[x]$ is a field. The most you can say in general is that $F$ being implies that $F[x]$ is an integral domain (and in fact, a Euclidean domain). In our example, we have $(R/I)[x] = \mathbb Z_2 [x]$, which is an integral domain, whereas $R[x] / (2, x) = \mathbb Z_2$, which is a field.