This is really just a yes/no question, but I need to make sure that I am right about the following:
If $\alpha$ is algebraic over $F$, then $F[\alpha]$ is a field. To show this, I let $f(x)$ be the minimal polynomial for $F$ for $\alpha$ over $F$. Then the ideal generated by $f$ is maximal. Then one can show that there is an isomorphism between $$ F[x] / \langle f(x)\rangle $$ and $$ F[\alpha]. $$ And this shows that $F[\alpha]$ is a field.
Is this a correct way to do this?
How might I prove that $F[\beta]$ is a field only if $\beta$ is algebraic over $F$?
Yes you are correct that that is a way to show $F[\alpha]$ is a field. To show that $F[\beta]$ is a field implies $\beta$ is algebraic, note that $\frac{1}{\beta}$ must be in $F[\beta]$, and thus is a polynomial in $\beta$. What can you do from here?