I'm having a technical problem evaluating the following integral:
$$\int_{r=0}^1\int_{\theta=0}^{\pi \over2} \cos^{2\epsilon -1}\theta \sin^{\epsilon-1}\theta e^{-ikr\sin^\epsilon\theta}d\theta dr$$ $\epsilon [0;\infty)$. Change of variable $t=\sin^\epsilon\theta$ brings the integral to the form (some constant coefficients are omitted):
$$\int_{r=0}^1\int_{t=0}^{1} (1-t^{2 \over \epsilon})^{\epsilon -1} e^{-ikrt}dt dr$$
My main concern is the integration of the expression above with respect to t. I wasn't able to evaluate this integral by parts/change of variable or using calculus of residues. Expanding $e^{-ikrt}=\sum_{n=0}^\infty (-ikrt)^n/n!$, then we have the integral of the type $$\int_{t=0}^{1} t^n(1-t^{2/\epsilon})^{\epsilon-1}dt$$ Integrating the expression above we get:$$\epsilon\Gamma(\epsilon)*\Gamma({\epsilon\over 2}\left(n+1)\right)\over2\Gamma\left({\epsilon\over 2}(n+3))\right)$$ and $$\int_{t=0}^{1} (1-t^{2 \over \epsilon})^{\epsilon -1} e^{-ikrt}dt =\sum_{n=0}^\infty{\epsilon\Gamma(\epsilon) \Gamma\left((n+1){\epsilon\over 2}\right)\over 2\Gamma(\left(n+3){\epsilon\over 2}\right)}{(-ikr)^n\over n!}$$ The last summation looks similar to the series expansion of hypergeometric function except it has $\epsilon/2$ in the argument of the gamma function and it seems that $\Gamma(kx)$ does not equal anything nice in terms of $\Gamma(x)$. Maybe I'm missing some clever change of variables or nice contour for residue calculation.
This integral arises when performing the Fourier transform of a superellipsoid.
EDIT
I decided to add a short connotation to this question. The original problem comes from physics and requires making a Fourier transform of a superellipsoid. I parametrized superellipsoid in the following manner: $$x=a*r*\cos^\eta\theta*\cos^\epsilon\phi,$$ $$y=b*r*\cos^\eta\theta*\sin^\epsilon\phi,$$ $$z=c*r*\sin^\eta\theta$$
Here $\sin^{\epsilon}\theta$ and $\cos^{\eta}\theta$ are signed functions i.e. $\sin^{\epsilon}\theta = sign(\sin\theta)*|\sin^{\epsilon}\theta|$.
The absolute value of the determinant of the Jacobian is $abc\epsilon\eta r^2\cos^{2\epsilon-1}\theta\sin^{\epsilon -1}\theta\cos^{\eta-1}\phi\sin^{\eta-1}\phi$.
Next, when performing Fourier transform direct vector k along z axis. Then we have in the region $\theta[0,\pi/2]$, $\phi[0,\pi/2]$: $$abc\epsilon\eta\int_{r=0}^1 \int_{\theta=0}^{\theta = \pi/2}\int_{\phi=0}^{\pi/2}e^{-ikr\sin^{\epsilon}}r^2\cos^{2\epsilon-1}\theta\sin^{\epsilon-1}\theta\cos^{\eta-1}\phi\sin^{\eta-1}\phi dr d\theta d\phi$$
Integration with respect to $\phi$ gives ${1 \over 2^{\eta - 1}}_2F_1({1 \over 2}, {2 - \eta \over 2};{3 \over 2};1)$ and we left with the integral of the form like in the very beginning of this question. Parameters $\epsilon$ and $\eta$ bear similar geometric meaning, $\epsilon = 0$, $\eta = 0$ gives a rectangular shape, as $\epsilon$ and $\eta$ approach 1 the shape becomes more like normal ellipse and high values of $\epsilon$ and $\eta$ give shapes with pointy cornets like stars, see the illustration at Paul Bourke's web page. My intuition suggests that Fourier transform should depend on $\epsilon$ in the similar way as it depends on $\eta$.
$\int_0^1\int_0^\frac{\pi}{2}\cos^{2\epsilon-1}\theta~\sin^{\epsilon-1}\theta ~e^{-ikr\sin^\epsilon\theta}~d\theta~dr$
$=\int_0^1\int_0^\frac{\pi}{2}\cos^{2\epsilon-2}\theta~\sin^{\epsilon-1}\theta ~e^{-ikr\sin^\epsilon\theta}~d(\sin\theta)~dr$
$=\int_0^1\int_0^1(1-x^2)^{\epsilon-1}x^{\epsilon-1}e^{-ikrx^\epsilon}~dx~dr$
$=\int_0^1\int_0^1x^{\epsilon-1}(1-x^2)^{\epsilon-1}\cos krx^\epsilon~dx~dr-i\int_0^1\int_0^1x^{\epsilon-1}(1-x^2)^{\epsilon-1}\sin krx^\epsilon~dx~dr$
$=\int_0^1\int_0^1\sum\limits_{n=0}^\infty\dfrac{(-1)^nk^{2n}r^{2n}x^{(2n+1)\epsilon-1}(1-x^2)^{\epsilon-1}}{(2n)!}~dx~dr-i\int_0^1\int_0^1\sum\limits_{n=0}^\infty\dfrac{(-1)^nk^{2n+1}r^{2n+1}x^{(2n+2)\epsilon-1}(1-x^2)^{\epsilon-1}}{(2n+1)!}~dx~dr$
$=\int_0^1\int_0^1\sum\limits_{n=0}^\infty\dfrac{(-1)^nk^{2n}r^{2n}x^\frac{(2n+1)\epsilon-1}{2}(1-x)^{\epsilon-1}}{(2n)!}~d(x^\frac{1}{2})~dr-i\int_0^1\int_0^1\sum\limits_{n=0}^\infty\dfrac{(-1)^nk^{2n+1}r^{2n+1}x^\frac{(2n+2)\epsilon-1}{2}(1-x)^{\epsilon-1}}{(2n+1)!}~d(x^\frac{1}{2})~dr$
$=\int_0^1\int_0^1\sum\limits_{n=0}^\infty\dfrac{(-1)^nk^{2n}r^{2n}x^{\frac{(2n+1)\epsilon}{2}-1}(1-x)^{\epsilon-1}}{2(2n)!}~dx~dr-i\int_0^1\int_0^1\sum\limits_{n=0}^\infty\dfrac{(-1)^nk^{2n+1}r^{2n+1}x^{(n+1)\epsilon-1}(1-x)^{\epsilon-1}}{(2n+1)!}~dx~dr$
$=\int_0^1\sum\limits_{n=0}^\infty\dfrac{(-1)^nk^{2n}r^{2n}B\left(\dfrac{(2n+1)\epsilon}{2},\epsilon\right)}{2(2n)!}~dr-i\int_0^1\sum\limits_{n=0}^\infty\dfrac{(-1)^nk^{2n+1}r^{2n+1}B((n+1)\epsilon,\epsilon)}{(2n+1)!}~dr$
$=\int_0^1\sum\limits_{n=0}^\infty\dfrac{(-1)^n\Gamma(\epsilon)\Gamma\left(\dfrac{(2n+1)\epsilon}{2}\right)k^{2n}r^{2n}}{2\Gamma\left(\dfrac{(2n+3)\epsilon}{2}\right)(2n)!}~dr-i\int_0^1\sum\limits_{n=0}^\infty\dfrac{(-1)^n\Gamma(\epsilon)\Gamma((n+1)\epsilon)k^{2n+1}r^{2n+1}}{\Gamma((n+2)\epsilon)(2n+1)!}~dr$
$=\left[\sum\limits_{n=0}^\infty\dfrac{(-1)^n\Gamma(\epsilon)\Gamma\left(\dfrac{(2n+1)\epsilon}{2}\right)k^{2n}r^{2n+1}}{2\Gamma\left(\dfrac{(2n+3)\epsilon}{2}\right)(2n+1)!}\right]_0^1-i\left[\sum\limits_{n=0}^\infty\dfrac{(-1)^n\Gamma(\epsilon)\Gamma((n+1)\epsilon)k^{2n+1}r^{2n+2}}{\Gamma((n+2)\epsilon)(2n+2)!}\right]_0^1$
$=\sum\limits_{n=0}^\infty\dfrac{(-1)^n\Gamma(\epsilon)\Gamma\left(\dfrac{(2n+1)\epsilon}{2}\right)k^{2n}}{2\Gamma\left(\dfrac{(2n+3)\epsilon}{2}\right)(2n+1)!}-i\sum\limits_{n=0}^\infty\dfrac{(-1)^n\Gamma(\epsilon)\Gamma((n+1)\epsilon)k^{2n+1}}{\Gamma((n+2)\epsilon)(2n+2)!}$
Which relates to Fox–Wright function