Let $G$ be a finite $p$-group. I'd like to prove (or disprove) that if the nilpotency class of $G$ equals two (i.e., $1 \neq G' \le Z$, where $Z$ is the center of $G$) and the exponent of $G$ equals four (i.e., $g^4 = 1$ for all $g \in G$), then the exponent of $G'$ equals two.
Note: By a formula $[gh, k] = [g, k]^h[h, k]$, we have $[gh, k] = [g, k][h, k]$ because $G' \le Z$. So $[g^2, h] = [g, h]^2$ in particular and our goal is equivalent to $G^2 \le Z$, where $G^{p^i} := \langle\, g^{p^i} \mid g \in G \,\rangle$.
I tried to prove it by showing $(G')^2 \le G^4$ but I couldn't prove it so far. As $G' \le G^2$ is true by a trick $[g, h] = (g^{-1})^2(gh^{-1})^2h^2$, I've been expecting something similar does the job. I also checked that the statement is true for $p$-groups of order dividing $2^6$ by a computer.
We have $$[g,h]^2 = (g^{-1}h^{-1}gh)^2 = (g^{-2}gh^{-2}hgh)^2.$$ Now collecting the terms $g^{-2}$ and $h^{-2}$ and the resulting commutators (which are central) to the left gives $$g^{-2}h^{-2}g^{-2}h^{-2}[g,h^{-2}][h^2,g^{-2}][g^3,h^{-2}](gh)^4 = g^{-2}h^{-2}g^{-2}h^{-2}[g,h^{-2}][h^2,g^{-2}][g^3,h^{-2}].$$ But commutators are bilinear in groups of class $2$, and $[g^2,h^2]=[g,h]^4=1$, so this simplifies to $g^{-4}h^{-4}[g^4,h^{-2}][h^2,g^{-2}]=1$.