Why is $$ \int \frac{\sin x (b-a\cos x)}{(b^2+a^2-2ab \cos x)^{3/2}}\,dx = \frac{a-b\cos x}{b^2 \sqrt{a^2-2ab\cos x + b^2}}\text{ ?}$$
Constant of integration omitted.
Why is $$ \int \frac{\sin x (b-a\cos x)}{(b^2+a^2-2ab \cos x)^{3/2}}\,dx = \frac{a-b\cos x}{b^2 \sqrt{a^2-2ab\cos x + b^2}}\text{ ?}$$
Constant of integration omitted.
On
The integral can be written as $$ \int \frac{\sin x (b-a\cos x)}{(a^2+b^2-2ab \cos x)^{\Large\frac{3}{2}}}\,dx=\int \frac{a\cos x-b}{\sqrt{(a^2+b^2-2ab \cos x)^3}}\,d(\cos x). $$ Let $y=\cos x$, then the integral becomes $$ \int \frac{ay-b}{\sqrt{(a^2+b^2-2ab y)^3}}\,dy=\int \frac{ay}{\sqrt{(a^2+b^2-2ab y)^3}}\,dy-\int \frac{b}{\sqrt{(a^2+b^2-2ab y)^3}}\,dy. $$ The right integral in RHS can easily be solved using substitution $u=a^2+b^2-2ab y$ and $du=-2ab\ dy$.
For the left integral in RHS, let $y=\dfrac{(a^2+b^2)}{2ab}\sin^2\theta$ and $dy=\dfrac{(a^2+b^2)}{ab}\sin\theta\cos\theta\ d\theta$, then \begin{align} \int \frac{ay}{\sqrt{(a^2+b^2-2ab y)^3}}\,dy&=\frac{\sqrt{a^2+b^2}}{2ab^2}\int \frac{\sin^3\theta}{\cos^2\theta}\,d\theta\\ &=\frac{\sqrt{a^2+b^2}}{2ab^2}\int \frac{\sin\theta(1-\cos^2\theta)}{\cos^2\theta}\,d\theta\\ &=\frac{\sqrt{a^2+b^2}}{2ab^2}\left[\int \frac{\sin\theta}{\cos^2\theta}\,d\theta-\int\sin\theta\ d\theta\right]\\ &=-\frac{\sqrt{a^2+b^2}}{2ab^2}\left[\int \frac{1}{\cos^2\theta}d(\cos\theta)+\int\sin\theta\ d\theta\right]. \end{align} From here, the last integral should be easy to evaluate. I leave it to you Kappa for the rest.
On
$\newcommand{\+}{^{\dagger}} \newcommand{\angles}[1]{\left\langle\, #1 \,\right\rangle} \newcommand{\braces}[1]{\left\lbrace\, #1 \,\right\rbrace} \newcommand{\bracks}[1]{\left\lbrack\, #1 \,\right\rbrack} \newcommand{\ceil}[1]{\,\left\lceil\, #1 \,\right\rceil\,} \newcommand{\dd}{{\rm d}} \newcommand{\down}{\downarrow} \newcommand{\ds}[1]{\displaystyle{#1}} \newcommand{\expo}[1]{\,{\rm e}^{#1}\,} \newcommand{\fermi}{\,{\rm f}} \newcommand{\floor}[1]{\,\left\lfloor #1 \right\rfloor\,} \newcommand{\half}{{1 \over 2}} \newcommand{\ic}{{\rm i}} \newcommand{\iff}{\Longleftrightarrow} \newcommand{\imp}{\Longrightarrow} \newcommand{\isdiv}{\,\left.\right\vert\,} \newcommand{\ket}[1]{\left\vert #1\right\rangle} \newcommand{\ol}[1]{\overline{#1}} \newcommand{\pars}[1]{\left(\, #1 \,\right)} \newcommand{\partiald}[3][]{\frac{\partial^{#1} #2}{\partial #3^{#1}}} \newcommand{\pp}{{\cal P}} \newcommand{\root}[2][]{\,\sqrt[#1]{\vphantom{\large A}\,#2\,}\,} \newcommand{\sech}{\,{\rm sech}} \newcommand{\sgn}{\,{\rm sgn}} \newcommand{\totald}[3][]{\frac{{\rm d}^{#1} #2}{{\rm d} #3^{#1}}} \newcommand{\ul}[1]{\underline{#1}} \newcommand{\verts}[1]{\left\vert\, #1 \,\right\vert} \newcommand{\wt}[1]{\widetilde{#1}}$ $\ds{\int{\sin\pars{x}\bracks{b - a\cos\pars{x}} \over \bracks{b^{2} + a^{2} - 2ab\cos\pars{x}}^{3/2}}\,\dd x ={a - b\cos\pars{x} \over b^{2} \root{a^{2} - 2ab\cos\pars{x} + b^{2}}};\ {\large ?}}$
With $\ds{t \equiv \cos\pars{x}}$: \begin{align} &\color{#00f}{\large% \int{\sin\pars{x}\bracks{b - a\cos\pars{x}} \over \bracks{b^{2} + a^{2} - 2ab\cos\pars{x}}^{3/2}}\,\dd x} =\int{b - at \over \pars{b^{2} + a^{2} - 2abt}^{3/2}}\,\pars{-\dd t} \\[3mm]&=\partiald{}{b}\int\pars{b^{2} + a^{2} - 2abt}^{-1/2}\,\dd t =\partiald{}{b}\bracks{\pars{b^{2} + a^{2} - 2abt}^{1/2} \over -ab} \\[3mm]&=-\,{1 \over a}\,{% \pars{1/2}\pars{b^{2} + a^{2} - 2abt}^{-1/2}\pars{2b - 2at}b -\pars{b^{2} + a^{2} - 2abt}^{1/2} \over b^{2}} \\[3mm]&=-\,{1 \over ab^{2}}\,{% \pars{b - at}b - \pars{b^{2} + a^{2} - 2abt}\over \root{b^{2} + a^{2} - 2abt}} =-\,{1 \over ab^{2}}\,{% abt - a^{2}\over \root{b^{2} + a^{2} - 2abt}} \\[3mm]&={1 \over b^{2}}\,{a - bt\over \root{a^{2} - 2abt + b^{2}}} =\color{#00f}{\large% {a - b\cos\pars{x}\over b^{2}\root{a^{2} - 2ab\cos\pars{x} + b^{2}}}} + \mbox{a constant} \end{align}
Since I spent so much time trying to work this out, I'm going to type the answer even though there are already two perfectly good answers. Hah!
Anyway, here's how to do it. Let $$u = b - a \cos x$$ and let $$y^2 = a^2 - b^2 + 2ub = a^2 + b^2 - 2ab \cos x.$$
Notice that then making the change of variables to $y$, we have that $$ \begin{align*} 2ydy = 2ab\sin x dx \Rightarrow \sin x dx =\dfrac{y \,dy}{ab}, &\quad b-a \cos x = u = \dfrac{y^2-a^2 + b^2}{2b}\\ \dfrac{1}{(a^2 - 2ab \cos x + b^2)^{3/2}} &= \dfrac{1}{y^3} \end{align*} $$ and therefore the integral is $$ \begin{align*} I &= \int \frac{\sin x (b-a\cos x)}{(b^2+a^2-2ab \cos x)^{3/2}} \,dx \\ &= \int\left( \dfrac{y \,dy}{ab}\right)\left( \dfrac{y^2-a^2 + b^2}{2b} \right)\left(\dfrac{1}{y^3}\right) \\ &= \dfrac{1}{2a b^2} \int \dfrac{y^2 - a^2 - b^2}{y^2} \, dy\\ &= \dfrac{1}{2ab^2} \left(\dfrac{y^2+a^2-b^2}{y}\right)+ \mbox{constant}\\ &= \dfrac{1}{2ab^2} \left(\dfrac{a^2 + b^2 - 2ab \cos x +a^2-b^2}{\sqrt{a^2 + b^2 - 2ab\cos x}}\right) + \mbox{constant}\\ &= \dfrac{a - b \cos x}{b^2 \sqrt{b^2+a^2-2ab \cos x}} + \mbox{constant} \end{align*} $$ as desired. Woohoo!