I'm having a difficult time solving this integral. I tried integrating by parts:
$\int\sin^2x\cos4x\,dx$
$u=\sin^2x$, $dv=\cos4x\,dx$
I used the power reducing formula to evaluate $\sin^2x$
$du = 2\sin x\cos x\,dx$, $v=1/4\sin4x$
$uv - \int\ v\,du$
$\dfrac{1}{4}\sin^2x\sin4x - \dfrac{1}{2}\int\sin x\cos x\sin4x\,dx$
After this step, I tried evaluating the integral by using the $\sin a\sin b$ property.
$\dfrac{1}{4}\sin^2x\sin4x + \dfrac{1}{4}\int\cos x(\cos5x-\cos3x)\,dx$
$$\begin{align*} \sin^2 x \cos 4x &= \sin x (\sin x \cos 4x) \\ &= \frac{1}{2} \sin x (\sin 5x - \sin 3x) \\ &= \frac{1}{4}(\cos 4x - \cos 6x + \cos 4x - \cos 2x) \\ &= \frac{1}{2} \cos 4x - \frac{1}{4} \cos 6x - \frac{1}{4} \cos 2x. \end{align*}$$
Consequently, we immediately and rather trivially obtain $$\int \sin^2 x \cos 4x \, dx = \frac{1}{8} \sin 4x - \frac{1}{24} \sin 6x - \frac{1}{8} \sin 2x + C.$$
Alternatively, observe $$\begin{align*} \sin^2 x \cos 4x &= \sin^2 x (\cos 3x \cos x - \sin 3x \sin x) \\ &= \cos 3x \sin^2 x \cos x - \sin 3x \sin^3 x \\ &= \frac{1}{3} \left( \frac{d}{dx}\left[\sin^3 x\right] \cos 3x + \frac{d}{dx} \left[ \cos 3x \right] \sin^3 x \right) \\ &= \frac{1}{3} \frac{d}{dx}\left[\cos 3x \sin^3 x\right], \end{align*}$$ the last step due to the product rule applied to the functions $\sin^3 x$ and $\cos 3x$; thus $$\int \sin^2 x \cos 4x \, dx = \frac{1}{3} \cos 3x \sin^3 x + C.$$ It is quite straightforward to demonstrate that these antiderivatives are equivalent.