I need to evaluate the following integral $$\int_{0}^{\pi/2} \cos^a (x) \sin (a x) \ dx,\ \ ~\ \ a\geq 0$$
I tried the following :$$I(a):=\int_{0}^{\pi/2} \cos^a (x) \sin (a x) \ dx$$ $$I(a)=\Im\left(\int_{0}^{\pi/2} \left(\frac{e^{ix}+e^{-ix}}{2}\right)^a e^{i a x} \ dx\right) $$ $$I(a)=\Im\left(2^{-a}\int_{0}^{\pi/2} \left(1+e^{-2ix}\right)^a e^{2i a x} \ dx\right) $$ $$I(a)=\Im\left(2^{-a}\int_{0}^{\pi/2} \left(1+e^{-2ix}\right)^a e^{i (2a+2) x} \ \frac{i}{2}d(e^{-2 i x})\right) $$ $u=e^{-2ix}$ ($u$ travels clockwise along the unit circle from $1$ to $-i$ to $-1$ then Cauchy's theorem lets us deform the countour to just under the real axis from $1$ to $0$ to $-1$) $$I(a)=\Im\left(2^{-a-1}\ i \int_{1}^{-1} (1+u)^a u^{-a-1} \ du\right) $$ $$I(a)=\Im\left(2^{-a-1}\ i \int_{1}^{0} (1+u)^a u^{-a-1} \ du\right)+\Im\left(2^{-a-1}\ i \int_{0}^{-1} (1+u)^a u^{-a-1} \ du\right) $$ $$I(a)=J(a)+K(a) $$ where $$J(a)=\Im\left(2^{-a-1}\ i \int_{1}^{0} (1+u)^a u^{-a-1} \ du\right) $$ and $$K(a)=\Im\left(2^{-a-1}\ i \int_{0}^{-1} (1+u)^a u^{-a-1} \ du\right)$$ $$J(a)=-2^{-a-1}\int_{0}^{1} (1+u)^a u^{-a-1} \ du $$ In $K(a)$, substitute $u=-v$ so that $du=-dv$ $$K(a)=\Im\left(-2^{-a-1}\ i \int_{0}^{1} (1-v)^a (-v)^{-a-1} \ dv\right)$$ $$K(a)=\Im\left((-1)^{-a}\ i \ 2^{-a-1} \int_{0}^{1} (1-v)^a v^{-a-1} \ dv\right)$$ Write $-1=e^{-i\pi}$ and $i=e^{i\pi/2}$ $$K(a)=\Im\left(e^{i\pi (a+1/2)} \ 2^{-a-1} \int_{0}^{1} (1-v)^a v^{-a-1} \ dv\right)$$ $$K(a)=\sin(\pi (a+1/2)) \ 2^{-a-1} \int_{0}^{1} (1-v)^a v^{-a-1} \ dv$$ $$K(a)=\cos(\pi a) \ 2^{-a-1} \int_{0}^{1} (1-v)^a v^{-a-1} \ dv$$ So by the definition of Beta function $$K(a)=\cos(\pi a) \ 2^{-a-1} \Gamma(a+1)\Gamma(-a)$$ By reflection formula of Gamma function $$K(a)=-\cos(\pi a) \ 2^{-a-1} \frac{\pi}{\sin \pi a}$$
$$I=\int_{0}^{\pi/2} \cos^a (x) \sin (a x) \ dx=\Im\int_{0}^{\pi/2} \cos^a (x) e^{ia x}~dx,\ \ \ \ a\geq 0$$
Let $z=e^{ix}$, and $I=\Im K$, where
$$K=\int_{0}^{\pi/2} \cos^a (x) e^{ia x}~dx=\frac1{2^a i}\int_{C_1}\left(\frac{1+z^2}{z}\right)^a\cdot\frac{z^a}{z}~dz$$
Define $$f(z)=\left(\frac{1+z^2}{z}\right)^a\cdot z^a$$
We use the following contour:
take the branch cut as shown in the figure.
$$\arg(z-i)\in (-\pi,\pi],~~~\arg(z)\in (-\pi,\pi],~~~\arg(z+i)\in (-\pi,\pi]$$
therefore,
$$\begin{align}\arg f&=a\left[\arg(z+i)+\arg(z-i)-\arg(z)\right]+a\arg(z)=a\left[\arg(z+i)+\arg(z-i)\right]\end{align}$$
we have
$$\frac1{2^a i}\int_{C_1}\frac{f(z)}z~dz+\frac1{2^a i}\int_{C_2}\frac{f(z)}z~dz+\cdots+\frac1{2^a i}\int_{C_5}\frac{f(z)}z~dz=0$$
The integral vanishes on $C_2$. On $C_4$, we have
$$\frac1{2^a i}\int_{C_4}\frac{f(z)}z~dz=\frac1{2^a i}\cdot \frac{-2\pi i}4 Res(z=0)=-\frac{\pi}{2^{a+1}}$$ On $C_3$, we have $\arg f=0+0=0$, $z=r e^{i\frac\pi2}$
$$\frac1{2^a i}\int_{C_3}\frac{f(z)}z~dz=\frac1{2^a i}\int_1^0\frac{(1-r^2)^a}{r}~dr=\frac i{2^a}\int_0^1\frac{(1-r^2)^a}{r}~dr$$
$$\frac1{2^a i}\int_{C_5}\frac{f(z)}z~dz=\frac1{2^a i}\int_0^1\frac{(1+r^2)^a}{r}~dr=-\frac i{2^a}\int_0^1\frac{(1+r^2)^a}{r}~dr$$
Therefore,
$$K+0+\frac i{2^a}\int_0^1\frac{(1-r^2)^a}{r}~dr-\frac{\pi}{2^{a+1}}-\frac i{2^a}\int_0^1\frac{(1+r^2)^a}{r}~dr=0\tag{1}$$ and
$$I=\Im K=\frac1{2^a}\int_0^1\frac{(1+r^2)^a}{r}~dr-\frac1{2^a}\int_0^1\frac{(1-r^2)^a}{r}~dr$$
Let $t=r^2$
$$I=\frac1{2^{a+1}}\int_0^1\frac{(1+t)^a-(1-t)^a}{t}~dt\tag{2}$$
Use binomial series expansion, $$I=\frac1{2^{a+1}}\int_0^1\frac1t\left(\sum_{k=0}^\infty\binom ak t^k-\sum_{k=0}^\infty \binom ak (-1)^kt^k\right)~dt$$
Even terms cancel out, we get
$$\begin{align}I&=\frac1{2^{a+1}}\int_0^1 \frac1t\cdot2\sum_{m=0}^\infty\binom{a}{2m+1} t^{2m+1}~dt=\frac1{2^{a}}\sum_{m=0}^\infty\binom{a}{2m+1}\int_0^1 t^{2m}~dt\end{align}$$
therefore,
$$\boxed{\int_{0}^{\pi/2} \cos^a (x) \sin (a x) \ dx=\frac1{2^{a}}\sum_{m=0}^\infty\frac1{2m+1}\binom{a}{2m+1}}$$
Numerically verified by Wolfram as below,
Byproduct
For eq.(1), if we take real part,
$$\Re K=\frac{\pi}{2^{a+1}}$$
we get the byproduct
$$\boxed{\int_{0}^{\pi/2} \cos^a (x) \cos (a x) \ dx=\frac{\pi}{2^{a+1}}}$$