Trying to prove the following: $$ \int_0^\infty xe^{-c x^2}\sinh(a x)\cos(bx)\,dx = \frac{1}{4}\sqrt{\dfrac{\pi}{c^3}}\exp\left\{\frac{a^2-b^2}{4c}\right\}\left(a\cos\left(\frac{ab}{2c}\right)-b\sin\left(\frac{ab}{2c}\right)\right) $$ How would one go about proving this?
PS A special case of this when $a=b$ is given in Gradsteyn & Ryzhik (7th ed), #4.146.4.
Assuming $a, b, c \gt 0$.
First extend the integral to the entire real line, as the integrand is clearly even. Then express the integral as
$$\begin{align}\int_0^{\infty} dx \, x \, e^{-c x^2} \sinh{a x} \, \cos{b x} &=\frac14 \Re{\left [\int_{-\infty}^{\infty} dx \, x \, e^{-c x^2} \, \left (e^{a x}-e^{-a x} \right ) e^{i b x}\right ]}\\ &=\frac14 \Re{\left [\int_{-\infty}^{\infty} dx \, x \, e^{-c x^2} \, e^{(a+i b) x} - \int_{-\infty}^{\infty} dx \, x \, e^{-c x^2} \, e^{-(a-i b) x}\right ]}\\ &= \frac14 \Re{\left [e^{(a+i b)^2/(4 c)}\int_{-\infty}^{\infty} dx \, x \, e^{-c (x-(a+i b)/(2 c))^2} \, \\ - e^{(a-i b)^2/(4 c)}\int_{-\infty}^{\infty} dx \, x \, e^{-c (x+(a-i b)/(2 c))^2} \right ]}\\ &= \frac14 \Re{\left [e^{(a+i b)^2/(4 c)}\int_{-\infty}^{\infty} dy \, \left (y+\frac{a+i b}{2 c} \right ) \, e^{-c y^2} \, \\ - e^{(a-i b)^2/(4 c)}\int_{-\infty}^{\infty} dy \, \left (y-\frac{a-i b}{2 c} \right ) \, e^{-c y^2} \right ]} \end{align} $$
Note that the integrals over $y\, e^{-c y^2}$ vanish. Thus we are left with simple integrals over $e^{-c y^2}$, which are $\sqrt{\pi/c}$. Thus we have that
$$\begin{align}\int_0^{\infty} dx \, x \, e^{-c x^2} \sinh{a x} \, \cos{b x} &= \frac14 \sqrt{\frac{\pi}{c}}\Re{\left [e^{(a+i b)^2/(4 c)} \frac{a+i b}{2 c} + e^{(a-i b)^2/(4 c)} \frac{a-i b}{2 c} \right ]}\\ &= \frac12 \sqrt{\frac{\pi}{c}} e^{(a^2-b^2)/4 c} \Re{\left [e^{i a b/(2 c)} \frac{a+i b}{2 c} \right ]}\\ &= \frac14 \sqrt{\frac{\pi}{c^3}}\,e^{(a^2-b^2)/4 c} \left [a \cos{\left (\frac{a b}{2 c} \right )}-b \sin{\left (\frac{a b}{2 c} \right )} \right ] \end{align}$$
as was to be shown. Note that the step in which we shifted the integral limits by a complex number is justified by Cauchy's theorem.