So, I'm having trouble getting the correct value for the integral $\int_0^{2\pi} \frac{\cos^2(3\theta)}{5-4\cos(2\theta)}\mathrm{d}\theta$. I substitute the exponential form of cosine into the equation, realize that $z=e^{i\theta}$, and deform the contour to be a unit circle. All of this yields:$$ -\frac{1}{4i}\oint \frac{(z^6+1)^2}{z^5(2z^2-1)(z^2-2)}\mathrm{d}z$$ This reveals 3 poles: a fifth order pole at $0$, and simple poles at $\frac{1}{\sqrt{2}}$, and $\sqrt{2}$. I decide to handle the simple pole $\frac{1}{\sqrt{2}}$ using Cauchy's integral formula, ignore the pole at $\sqrt{2}$ because it is outside of the contour, and to find the residue using a Laurent series of the pole at $0$.
Firstly, the pole at $\frac{1}{\sqrt{2}}$. I use $u=z^2$ to obtain the proper form for the integral formula. This gives me:$$ -\frac{1}{16i} \oint \frac{(u^3+1)^2}{\left(u-\frac{1}{2}\right)(u-2)u^3}\mathrm{d}u$$ Using Cauchy's integral formula, I obtain that this integral is equal to $$2\pi i *f\left(\frac{1}{2}\right)=\frac{-\left(\frac{81}{64}\right)2\pi i}{-\left(\frac{3}{2}\right)\left(\frac{1}{8}\right)16i}=\frac{27\pi}{32}$$
Next, I find the residue using a Laurent series at $0$: \begin{align} f(z) &= -\frac{1}{4i}\left(\frac{(z^6+1)^2}{z^5(2z^2-1)(z^2-2)}\right) = -\frac{1}{8i} \left(\frac{(z^6+1)^2}{z^5(1-z^2)\left(1-\frac{z^2}{2}\right)}\right)\\ &= -\frac{(z^6+1)^2}{8z^5i}(1+2z^2+4z^4+...)\left(1+\frac{z^2}{2}+\frac{z^4}{4}+...\right) \end{align} Only being concerned with the terms with factors of $z^4$ (to obtain the $\frac{1}{z}$ term), and disregarding $(z^6+1)^2$ as it is also unimportant to the term:$$-\frac{1}{8z^5i}z^4\left(\frac{1}{4}+1+4\right)=-\frac{21}{32i}$$ Multiplying by $2\pi i$ to obtain the residue we obtain $$-\frac{21\pi}{16}$$
Summing the residues I get $$\frac{27\pi}{32}-\frac{21\pi}{16}=-\frac{15\pi}{32}$$ which is incorrect, as it should be $\frac{3\pi}{8}$. If anyone could point out where my error is, I would be very grateful. I've been going over this problem for 8 hours now and it's driving me crazy.