For $a>b>-1$, I want to evaluate $$\int_{C} z^{a-1} \left(z + z^{-1}\right)^{b} \, dz$$ where $C$ is the right half of the circle $|z|=1$ traversed counterclockwise.
So I closed the contour with the vertical line segment $[-i, i]$ (indented at $z=i$, $z=0$, and $z=-i$).
Under the restriction mentioned above, I get
$\begin{align} \int_{C}z^{a-1}\left( z+z^{-1}\right)^{b} \, dz &= \int_{-1}^{1}(it)^{a-1}\left( it+(it)^{-1}\right)^{b} \, i \, dt \\ &=\int_{-1}^{1}(it)^{a-1}\left(\frac{1-t^{2}}{it}\right)^{b} \, i \, dt \\ &= \require{cancel} \cancel{i} \ i^{a-b}\int_{-1}^{1}t^{a-b-1}\left(1-t^{2}\right)^{b} \, dt \\ &=\cancel{i} \left[\cos\left(\frac{\pi(a-b)}{2}\right)+i\sin\left(\frac{\pi(a-b)}{2}\right)\right]\int_{-1}^{1}t^{a-b-1}\left(1-t^{2}\right)^{b} \, dt. \end{align}$
But the book says that it should be $$ 2i \sin\left(\frac{\pi(a-b)}{2}\right)\int_{0}^{1}t^{a-b-1}\left(1-t^{2}\right)^{b}\ dt.$$
How do we get that?
OK, I think I can explain this...first of all, you made a mistake in the 3rd line, so let's take it form there. The integral you seek is
$$\begin{align}i^{a-b} \int_{-1}^1 dt \: t^{a-b-1} (1-t^2)^b &= e^{i \pi (a-b)/2} \left [ \underbrace{\int_{-1}^0 dt \: t^{a-b-1} (1-t^2)^b}_{t=u e^{-i \pi}} + \int_{0}^1 dt \: t^{a-b-1} (1-t^2)^b \right ]\\ &= e^{i \pi (a-b)/2} \left ( e^{-i \pi (a-b-1)}+1\right ) \int_{0}^1 dt \: t^{a-b-1} (1-t^2)^b \\ &= \left ( -e^{-i \pi (a-b)/2}+ e^{i \pi (a-b)/2}\right ) \int_{0}^1 dt \: t^{a-b-1} (1-t^2)^b\end{align}$$
which brings you to your result.