I am doing Stein's complex analysis chapter 6 exercise 10(a), which stated as follows:
An integral of the form $$F(z)=\int_{0}^{\infty}f(t)t^{z-1}dt$$ is called a Mellin transformation, and we shall write $\mathcal{M}(f)(z)=F(z)$. For example, the gamma function is the Mellin transform of the function $e^{-t}$.
Prove that $$\mathcal{M}(\cos)(z)=\int_{0}^{\infty}\cos(t)t^{z-1}dt=\Gamma(z)\cos\Bigg(\pi\dfrac{z}{2}\Bigg)\ \ \text{for}\ \ 0<\Re(z)<1,$$ and $$\mathcal{M}(\sin)(z)=\int_{0}^{\infty}\sin(t)t^{z-1}dt=\Gamma(z)\sin\Bigg(\pi\dfrac{z}{2}\Bigg)\ \ \text{for}\ \ 0<\Re(z)<1.$$
I am really close to finish the proof but got stuck in the end. Here is my attempt:
Using the hint provided by Stein (which is just to consider some contour integral, not related to my confusion), I got
$$\Gamma(z)i^{-z}=\int_{0}^{\infty}\cos(t)t^{z-1}dt-i\int_{0}^{\infty}\sin(t)t^{z-1}dt.$$
I can now prove it for real $0<z<1$:
Note that $i^{-z}=e^{-\frac{\pi}{2z}}=\cos(\frac{\pi}{2z})-i\sin(\frac{\pi}{2z}),$ so that $$\Gamma(z)\cos\Bigg(\dfrac{\pi}{2z}\Bigg)-i\Gamma(z)\sin\Bigg(\dfrac{\pi}{2z}\Bigg)=\int_{0}^{\infty}\cos(t)t^{z-1}dt-i\int_{0}^{\infty}\sin(t)t^{z-1}dt.$$ Compare real part and imaginary part separately, we have the desired identity: $$\Gamma(z)\cos\Bigg(\dfrac{\pi}{2z}\Bigg)=\int_{0}^{\infty}\cos(t)t^{z-1}dt\ \ \text{and}\ \ \Gamma(z)\sin\Bigg(\dfrac{\pi}{2z}\Bigg)=\int_{0}^{\infty}\sin(t)t^{z-1}dt.$$
My question is how to extend this result to complex $0<\Re(z)<1$. Using analytic continuation?
Honestly even though I know the definition of analytic continuation, I am always confused about how to really conduct it. Seems like books and literatures (including Stein) directly said something like "it suffices to prove ...., and the general result follows immediately from analytic continuation" without really telling me how to analytically continue it.
I tried to bypass the need for analytic continuation, but it seems that when $z\in\mathbb{C}$, we cannot have the nice identity for $i^{-z}$, and I will end up with a wrong identity.
What should I do?