Help to find complex Fourier series coefficient of this periodic function

Question

I'm having big trouble finding the complex Fourier series coefficient of the following periodic function

$$\frac{a-b\cos\varphi}{\sqrt{a^2+b^2-2ab\cos\varphi}}$$

Mathematica is unable to compute it!!

Answer 1

$$\frac{1}{\pi} \int_0^{\pi} \frac{a-b\cos\varphi}{\sqrt{a^2+b^2-2ab\cos\varphi}} \cos n \varphi\, d\varphi = \, _3\tilde{F}_2\left(\tfrac{1}{2},\tfrac{1}{2},1;1-n,n+1;\tfrac{4 a b}{(a+b)^2}\right)$$ $$\qquad\qquad\qquad\qquad-\frac{ b}{a+b} \, _3\tilde{F}_2\left(\tfrac{1}{2},\tfrac{3}{2},2;2-n,n+2;\tfrac{4 a b}{(a+b)^2}\right),$$ with $\tilde{F}$ defined here.

Answer 2

You might combine the Legendre expansion (wlog $a>b$) $$ \frac1{\sqrt{a^2+b^2-2ab\cos\varphi}}=\frac1a\sum_{n=0}^\infty\left(\frac ba\right)^nP_n(\cos\varphi) $$ with the Fourier expansion (Gradshteyn-Ryzhik 8.826) $$ P_n(\cos\varphi)=\\\frac{2^{n+2}n!}{\pi(2n+1)!!}\left(\sin(n+1)\varphi+\frac11\frac{n+1}{2n+3}\sin(n+3)\varphi+\frac11\frac32\frac{n+1}{2n+3}\frac{n+2}{2n+5}\sin(n+5)\varphi+\dots\right) $$ and product-to-sum formulas when multiplying by $a-b\cos\varphi$.