I am trying to integrate $$\int \frac{dx}{\sqrt{(a-x)(x-b)(x^2+q+px)}},$$ where $a>b$, and $p$ and $q$ are real positive constants.
I would be very grateful if I can get some hints on how to proceed. Wolfram Mathematica tells me that this will be giving me an ellpitic integral.
Yes, it has the structure of an Elliptic integral. You cannot solve it in terms of elementary functions, so you have to accept W. Mathematica's result, which is the only "true" result you can get.
By the way, for the curious people, the result is:
$$\frac{2 (a-x) \left(a \left(x-\sqrt{-\text{px}-q}\right)+x \sqrt{-\text{px}-q}+\text{px}+q\right) \sqrt{\frac{a \left(-x \sqrt{-\text{px}-q}+\text{px}+q\right)-(\text{px}+q) \left(\sqrt{-\text{px}-q}+x\right)}{(a-x) (\text{px}+q)}} \sqrt{\frac{(b-x) \left(a^2+\text{px}+q\right)}{(a-x) \left(a \left(b-\sqrt{-\text{px}-q}\right)+b \sqrt{-\text{px}-q}+\text{px}+q\right)}} F\left(\sin ^{-1}\left(\frac{\sqrt{\frac{(\text{px}+q) \left(\sqrt{-\text{px}-q}-x\right)+a \left(\text{px}+q+x \sqrt{-\text{px}-q}\right)}{(\text{px}+q) (a-x)}}}{\sqrt{2}}\right)|-\frac{2 (a-b) \sqrt{-\text{px}-q}}{\text{px}+q+a \left(b-\sqrt{-\text{px}-q}\right)+b \sqrt{-\text{px}-q}}\right)}{\left(a^2+\text{px}+q\right) \sqrt{\frac{a \left(x \sqrt{-\text{px}-q}+\text{px}+q\right)+(\text{px}+q) \left(\sqrt{-\text{px}-q}-x\right)}{(a-x) (\text{px}+q)}} \sqrt{(a-x) (x-b) \left(\text{px}+q+x^2\right)}}$$
EDIT
As suggested by @Claude Leibovici, one can write $x^2+px+q$ as $(x-c)(x-d)$ (in a way such that $q = cd$ etc...), and get the more suitable solution:
$$\frac{2 (x-b) (x-d) \sqrt{\frac{(a-b) (c-x)}{(a-x) (c-b)}} F\left(\sin ^{-1}\left(\sqrt{\frac{(a-d) (b-x)}{(b-d) (a-x)}}\right)|\frac{(a-c) (b-d)}{(b-c) (a-d)}\right)}{(b-d) \sqrt{\frac{(a-b) (a-d) (b-x) (x-d)}{(a-x)^2 (b-d)^2}} \sqrt{(a-x) (x-b) (x-c) (x-d)}}$$
EDIT 2
Due to some considerations, we can simplify the non Elliptic terms to get
$$-\frac{2 (b-x) (x-d) \sqrt{\frac{x-c}{(x-a) (c-b)}}}{\sqrt{\frac{(a-d) (x-b) (d-x)}{(a-x)^2}} \sqrt{(x-a) (b-x) (x-c) (x-d)}} \times \text{Elliptic Term}$$
Due to the unknown nature of $x$ and of $a$ wrt $b$ and $c$ wrt $d$, we cannot simplify it more, I believe.
EDIT 3 - A Very Special Case
If we assume that $x>c$, $x>a$, $c>b$, $d>a$, we get
$$\frac{-2\text{sgn}(d-x) F\left(\sin ^{-1}\left(\sqrt{\frac{(a-d) (b-x)}{(b-d) (a-x)}}\right)|\frac{(a-c) (b-d)}{(b-c) (a-d)}\right)}{\sqrt{(a-d) (-(b-c))}}$$