Good afternoon,
consider the integral function $f(x):=\int_{0}^{+\infty}\mathrm{d}t\,\frac{\cos(xt)}{\sqrt{t^2+1}}$. I would like to derive $f$ with respect to $x$ to obtain $\frac{\mathrm{d}f(x)}{\mathrm{d}x}=-\int_{0}^{+\infty}\mathrm{d}t\,\frac{t\sin(xt)}{\sqrt{t^2+1}}$, but the integral is not absolutely convergent and Lebesgue's dominated convergence theorem cannot be applied here to justify the derivation under the integral sign. How could one proceed here?
The integral $\int_{0}^{\infty} \frac{t \sin(xt)}{\sqrt{t^{2}+1}} \, \mathrm dt$ doesn't converge.
However, we can put $f(x)$ in a form that allows us to differentiate under the integral sign.
Assume that $x>0$.
Notice that $$\begin{align} f(x) &= \int_{0}^{\infty} \cos(xt) \left(\frac{1}{\sqrt{1+t^{2}}}- \frac{\boldsymbol{1}_{t > 1}}{t} \right) \, \mathrm dt + \int_{1}^{\infty} \frac{\cos xt}{t} \, \mathrm dt \\ &=\int_{0}^{\infty} \cos(xt) \left(\frac{1}{\sqrt{1+t^{2}}}- \frac{\boldsymbol{1}_{t > 1}}{t} \right) \, \mathrm dt - \operatorname{Ci}(x), \end{align}$$ where $\operatorname{Ci}(x)$ is the cosine integral function.
Differentiating both sides, we get $$\begin{align} \frac{\mathrm d}{\mathrm dx} f(x) &= -\int_{0}^{\infty}\sin(xt) \left(\frac{t}{\sqrt{1+t^{2}}}- \boldsymbol{1}_{t > 1} \right) \, \mathrm dt - \frac{\cos(x)}{x} \\ &= -\int_{0}^{\infty} \sin(xt) \left(\frac{t}{\sqrt{1+t^{2}}}-1 \right) \, \mathrm dt - \frac{1}{x} \end{align}$$
since $\int_{0}^{1} \sin(xt) \, \mathrm dt = \frac{1- \cos(x)}{x}$.
Differentiating under the integral sign was permissible since $\left| -\sin(xt) \left( \frac{t}{\sqrt{1+t^{2}}}- \boldsymbol{1}_{t > 1}\right) \right|$ is dominated on $[0,1]$ by $1$ and dominated on $(1, \infty)$ by $\left(1- \frac{t}{\sqrt{1+t^{2}}} \right)$.
And since $f(x)$ is an integral representation of $K_{0}(x)$, we have
$$\frac{\mathrm d}{\mathrm dx}K_{0}(x) = - K_{1}(x) = - \int_{0}^{\infty}\sin(xt) \left(\frac{t}{\sqrt{1+t^{2}}}-1 \right) \, \mathrm dt - \frac{1}{x}. $$