What about the converse?
Proof:
Let $\alpha$'s minimal polynomial is $p(x)$, then $p(x)=(x-\alpha)f(x)$, for some $f(x)\in F[x]$.
$p(x)=(x-\alpha)f(x)=xf(x)-\alpha f(x)$, choose $\style{text-decoration:line-through}{f(x)=1}$.
$\style{text-decoration:line-through}{x-\alpha\in F[x]\implies (-\alpha)\in F \implies (\alpha) \in F \implies (\alpha)(\alpha)\in F \implies \alpha^2\in F\implies -\alpha^2\in F.}$
We also have $\style{text-decoration:line-through}{x \in F[x]}$ and $\style{text-decoration:line-through}{F[x]}$ is ring. So it must be closed under addition.
$\style{text-decoration:line-through}{\implies x-\alpha^2 \in F[x].}$
What do you think?
The converse is true, but I think there is a mistake in your proof. If you can write $p$ as $(x-α)*f$ for for some $f(x)∈F[x]$, then the product of the leading coefficient of $f$ and $α$ lies in $F$.
Thus $α$ lies in $F$. But you want α to be an arbitrary. Let me give a proof for the statement:
If α is algebraic over $F$, then the field extension $F \subset F[\alpha]$ is finite.
Thus every $\beta \in F[\alpha]$ is algebraic. But $\alpha ^2$ is an element of $F[\alpha]$.