One could show the integral is uniformly convergent by definition of Riemann integration.
Let division $D(n)=x_0, x_1,\dots, x_n\dots$, where$x_i=\frac1a(\frac{i}{2n}\pi)=\frac1a(\lambda_i+2k\pi)$, $\lambda_i=\frac{i\ \text{mod}( 4n)}{2n}\pi, k =[\frac i{4n}]$.
The upper sum $S_n=\lim_{n\to\infty}\sum_{i=0}^\infty \frac1n M_i =\lim_{n\to\infty}\frac1n[\dots+\sum_{i=-2n}^{-n-1} \frac{\cos ax_{i+1}}{1+{x_{i+1}}^2} +\sum_{i=-n}^{-1} \frac{\cos ax_{i+1}}{1+{x_i}^2} +\sum_{i=0}^{n-1} \frac{\cos ax_i}{1+{x_i}^2} +\sum_{i=n}^{2n-1} \frac{\cos ax_i}{1+{x_{i+1}}^2}+\dots].$
Similarly we have the lower sum $s_n$. Obviously, $S_n-s_n$ has terms such as
$\frac{\cos ax_{i+1}}{1+{x_{i+1}}^2}- \frac{\cos ax_{i}}{1+{x_{i}}^2} \quad \text{(using Taylor expansion)}\\ =\frac{\left\{\left[\cos ax_i(1-\frac{(\frac{\pi}{2n})^2}{2!}+O(\frac1{n^4}))-\sin ax_i ((\frac{\pi}{2n})-\frac{(\frac{\pi}{2n})^3}{3!}+O(\frac1{n^5}))\right](1+{x_{i}}^2) -\cos ax_i\left(1+{x_i}^2+2x_i\frac\pi{2an}+(\frac\pi{2an})^2\right)\right\} (1-\frac{2x_i}{(1+{x_i}^2)}\frac\pi{2an}+O(\frac1{n^2}))} {(1+{x_i}^2)(1+{x_{i}}^2)}\\ =\frac{\frac{\pi}{2n}\left\{|\sin ax_i| (1+{x_{i}}^2)+|\cos ax_i|2x_i\frac1a\right\}+O(\frac1{n^2})} {(1+{x_{i}}^2)^2}.$
With similar calculation, we have $S_n-s_n= \frac{\pi}{2n^2}\sum_{i=0}^{\infty} \frac{|\sin ax_i|} {(1+{x_{i}}^2)} +\frac{|\cos ax_i|2x_i\frac1a} {(1+{x_{i}}^2)^2}+O(\frac1{n})\\ =\frac{\pi}{2n^2}\sum_{k=0}^\infty\sum_{i=0}^{4n-1} \frac{|\sin \lambda_i|} {(1+{[\frac1a(\lambda_i+2k\pi)]}^2)} +\frac{|\cos \lambda_i|2[\frac1a(\lambda_i+2k\pi)]\frac1a} {(1+{[\frac1a(\lambda_i+2k\pi)]}^2)^2}+O(\frac1{n})\\ <\frac1n\sum_{k=0}^\infty\sum_{i=0}^{4n-1} \frac{\frac{\pi}{2n}|\sin \lambda_i|} {(1+{[\frac1a(2k\pi)]}^2)} +\frac{\frac{\pi}{2n}|\cos \lambda_i|2[\frac1a(2\pi+2k\pi)]\frac1a} {(1+{[\frac1a(2k\pi)]}^2)^2}+O(\frac1{n}) \quad (\text{using defintion of integration})\\ <\frac1n\sum_{k=0}^\infty \frac{4} {(1+{[\frac1a(2k\pi)]}^2)} +\frac{4\cdot2[\frac1a(2\pi+2k\pi)]\frac1a} {(1+{[\frac1a(2k\pi)]}^2)^2}+O(1),$
How does one then prove the uniform convergence?
Write $\sum_{k=K}^{K+p}$ for $\sum_{k=0}^\infty$, then uniform convergence implies -- for integration on a tail is smaller than $\epsilon$, the tail ($[K, +\infty)$) defined independently of $a$; and so are upper and lower sums $S_n, s_n$ over this tail (and $N$ is independent of $a$ too), right?, -- that, $\forall \epsilon, \exists K, N \text{, both independent of $a$, such that }\forall p>0, n>N$, the above expression is less than $\epsilon$.
It seems the O(1) term makes it divergent. We could also find for however large $K, a=K^2$ such that one term in $\sum_{k=K}^{K+p}>\frac4{1+4\pi^2}$ which is not arbitrarily small, and so the series doesn't converge uniformly, right?
Edited to add: Below I gave a simpler proof inspired by an answer and tried to answer part of my questions above.
Another Q: Perhaps the integral is not uniform convergent but still a continuous function of $a$?
This is a sketch of solution based on the notion of Riemann improper integrals, which seems to be what the OP is referring to (the interval of integration is infinite).
Let $\varepsilon>0$. Fix $a_0$.
Since $\frac{1}{1+x^2}$ is integrable (in the sense that $\lim_{c\rightarrow\infty}\int^c_0\frac{dx}{1+x^2}$), there is $C$ such that $$\int^{c'}_c\frac{dx}{1+x^2}<\frac{\varepsilon}{2}$$ for all $c'>c>C$.
Since $\cos$ is periodic and continuous, it is also uniformly continuous; hence, there is $\delta>0$ such that if $|b-b'|<\delta$, then $|\cos b - \cos b'|<\frac{\varepsilon}{2\pi}$
Putting things together, if $c>C$ and $|a-a_0|C<\delta$, then
$$ \Big|\int^c_0\frac{\cos(ax)-\cos(a_0x)}{1+x^2}\,dx\Big|\leq\int^C_0\frac{|\cos(ax)-\cos(a_0x)|}{1+x^2}+\int^c_C\frac{dx}{1+x^2}<2\varepsilon $$
Edit: Just to complete the argument a little further:
Letting $c\rightarrow\infty$ implies that
$$ \Big|\int^\infty_0\frac{\cos(ax)-\cos(a_0x)}{1+x^2}\,dx\Big|\leq2\varepsilon $$ whenever $|a-a_0|<\frac{\delta}{C}$, and the conclusion follows.