This is likely a very basic question, but I have not been able to find an answer (or even someone posing this question), so here goes.
Let $P_k(n) = \sum_{m=1}^n m^k$. It is well-known that
$$\displaystyle P_3(n) = \left(\frac{n(n+1)}{2}\right)^2.$$
In particular, $P_3(n)$ is a perfect square as a polynomial in $n$. Is $k = 3$ the only $k$ for which this is the case? That is, does there exist $k \geq 4$ such that $P_k(n)$ is a perfect $\ell$-th power for some $\ell \geq 2$?
There is no other $k$.
Faulhaber's formula for $F_k(x)$ has rational coefficients and leading term $1/(k+1)$. If $F_k(x)$ is the square of a polynomial, we have $$ F_k(x) = \frac{g(x)^2}{k+1}$$ where $g(x)^2$ is monic with rational coefficients. Now $g(x)$, being the square root of a monic polynomial with rational coefficients, has rational coefficients (in fact the square root of a monic polynomial over the rationals can be written as a Laurent series over the rationals; it is a polynomial if the terms in negative powers of $x$ are all $0$).
In particular, $$ 1 + 2^k = \frac{F_k(2)}{F_k(1)} = \frac{g(2)^2}{g(1)^2} $$ is the square of a rational number, and thus the square of a positive integer $m$. But $1 + 2^k = m^2$ means $(m-1)(m+1) = 2^k$, so both $m-1$ and $m+1$ are powers of $2$. The only powers of $2$ that differ by $2$ are $2$ and $2^2$, which gives us the case $k=3$.