Under what conditions does every automorphism of a finite extension restrict to a base field automorphism?

Question

Suppose that $F$ is a field of characteristic $0$, and $E$ is a finite extension of $F$. Then my question is, under what conditions does every automorphism of $E$ restrict to an auomorphism of $F$?

I know that this holds if $F=\mathbb{Q}$, but I want to keep $F$ arbitrary and impose a condition on $E$.