integrate product of trig functions

Question

I need to find the Fourier cosine series for $\cos(3x)\sin^2(x)$, But I don't even know where to start to determine $$\int _0^{\pi }\cos(3x)\sin^2(x)\cos(k x)dx$$

Answer 1

$$ \sin^2 x = \frac12 - \frac12\cos(2x). $$ $$ \cos(3x)\left(\frac12 - \frac12\cos(2x)\right) = \frac12\cos(3x) - \frac12\cos(3x)\cos(2x). $$

\begin{align} & \cos(3x)\cos(2x) - \sin(3x)\sin(2x) = \cos(5x) \\ & \cos(3x)\cos(2x) + \sin(3x)\sin(2x) = \cos(x) \\ \\ \text{Therefore } & \cos(3x)\cos(2x) = \frac{\cos(5x)+\cos(x)}{2} \end{align}

Answer 2

Hint: You can express $\sin^2 x=\frac 12 - \frac 12 \cos 2x$. The first term will give zero unless $k=3$. For the second term you should be able to express $\cos 3x \cos 2x$ as a sum of the from $a \cos 5x + b \cos x$ and then argue you need $k=1$ or $5$ to not get zero.