$$=\int^\infty_{-\infty} e^{-i\xi x} f(ax+b) \, dx$$
Let $ax-b=u$
$$=\frac{1}{a} e^{-i\xi \frac b a} \int^\infty_{-\infty} e^{-i\frac{\xi}{a} x} f(u) \, du$$
$$=\frac{1}{a} e^{-i\xi \frac b a} F\left(\frac \xi a\right) \quad \text{(inverse Fourier transform)}$$
But the answer is
$$=\frac{1}{|a|} e^{-i\xi \frac{b}{a}} F(\frac{\xi}{a}),$$
why the modulus in $a$?
If you substitute $ax−b=u$ then the limits of the integral change to "$a\cdot(-\infty)$" and "$a\cdot\infty$" so if $a$ is negativ you get a $\infty$ as a lower limit and $-\infty$ on top.
If you change them back you will have a minus in front of the integral which you can put before the a so you have $-a$ if $a$ is negative and $a$ if a is positive… so $|a|$