If $G$ is square free such that $|G|=p_1\cdots p_n$, where $p_n>p_{n-1}>\cdots$. Then we can use the N/C Theorem, along with some induction, to show that the Sylow $p_n$-subgroup of $G$, $P_n$, is normal in $G$.
Suppose $G$ isn't square free, and let's assume $G$ is solvable. Suppose $p_k$ is the largest prime divisor of $G$ with exponent $1$. Could we necessarily say $P_k\trianglelefteq G$?
The answer is no, as noted in a comment by Steve D. For example, consider $G = A_4$, which has order $2^2 \cdot 3$ but does not have a normal Sylow $3$-subgroup.
In fact a finite solvable group does not need to have any normal Sylow subgroups, consider for example $G = S_4$.
But the answer is yes for some classes of solvable groups. For example besides groups of squarefree order, more generally the property you ask about holds for supersolvable finite groups.
That is, if $G$ is a finite supersolvable group and $p$ is the largest prime divisor of $|G|$, then $G$ has a normal $p$-Sylow.