I have been looking around and found similar problems but I think I'm making a mistake specific to this problem. For some reason I keep finding $0$ using Residue Theorem instead of what I am suppose to find
show that $\int_{0}^{2\pi}\frac{1}{a+b\cos{(\theta})}d\theta = \frac{2\pi}{\sqrt{a^2-b^2}} $
using $z=e^{i\theta}\rightarrow d\theta=\frac{dz}{iz}$, and rewriting $cos(\theta)=\frac{z+z^{-1}}{2}$
I changed the original integral to $\oint\frac{-2i}{bz^2+2az+b}dz$
Then I found the zeros of the polynomial in the denominator to be
$z_1=\frac{-a+\sqrt{a^2-b^2}}{b}$, $z_2=\frac{-a-\sqrt{a^2-b^2}}{b}$
Then the integral becomes $\oint\frac{-2i}{(z-z_1)(z-z_2)}dz$ and we have simple poles at $z_1,z_2$ so I start using the Residue Theorem.
=$2\pi i\left[Res(f(z),z_1)+Res(f(z),z_2)\right]$
=$2\pi i\left[\lim\limits_{z\to z_1}(z-z_1)\frac{-2i}{(z-z_1)(z-z_2)}+\lim\limits_{z\to z_2}(z-z_2)\frac{-2i}{(z-z_1)(z-z_2)}\right]$
=$2\pi i\left[\lim\limits_{z\to z_1}\frac{-2i}{(z-z_2)}+\lim\limits_{z\to z_2}\frac{-2i}{(z-z_1)}\right]$
=$4\pi\left[\frac{1}{z_1-z_2}+\frac{1}{z_2-z_1}\right]$, when plugging in the values for $z_1,z_2$ I find
=$0\neq \frac{2\pi}{\sqrt{a^2-b^2}}$
EDIT:The original problem is stated like this
Show that
$\int_{0}^{2\pi}\frac{1}{a+b\cos{(\theta)}}d\theta=\frac{2\pi}{\sqrt{a^2-b^2}}$
Hint: Use the substitution $z=e^{i\theta}$
You need to use the residues only of poles lying inside the contour. Here the contour is the unit circle and the quadratic denominator, whose coefficients read the same fowards and backwards, must have its roots as reciprocals of one another. Therefore just one pole is inside the unit circle (for $|a|>|b|$, which is required for the integral to converge).
You need to identify the correct pole inside the contour and take its residue alone, which will be nonzero. For example, with $a$ and $b$ both positive and $a>b$, you find that $z_2<-1$ (the numerator there is less than $-b$), so the reciprocal, $z_1$, lies between $0$ and $-1$ — so the latter is the residue inside the unit circle.
You may want to check whether there is an assumption that $a$ is positive. That is needed for the given answer. A negative value of $a$ would give a negative instead of positive value for the definite integral.