I have 3 questions here, mainly related to convergence and justifying interchanging the order of differentiation/integration and summation. The authors seem to think these justifications are trivial but I have had problems with them.
Here they say the double sum converges for $s > 1$, but I haven't been able to prove it. (The sum is over primes $p$ and all positive integers $m$).
They differentiate here term by term and assume the resulting series converges to the first quantity on the left.
In the first step here they interchange the order of summation and integration.



Now, since the $k^{\text{th}}$ prime is greater than $k$
$$\sum_{\substack{p \text{ prime} \\ m \geq 1}} \frac{1}{p^{mc}m} \leq \sum_{\substack{k \geq 2 \\ m \geq 1}} \frac{1}{k^{mc}m} \leq \sum_{\substack{k \geq 2 \\ m \geq 1}} \frac{1}{k^{mc}}.$$
Finally,
\begin{align*} \sum_{\substack{k \geq 2 \\ m \geq 1}} \frac{1}{k^{mc}} &=\sum_{k \geq 2} \left( \frac{1}{1-\frac{1}{k^{c}}}-1 \right) \\ &=\sum_{k \geq 2} \frac{1}{k^{c}-1} \\ &<\infty. \end{align*}
By the Weierstrass $M$-test, this computation implies that $\sum \frac{p^{-ms}}{m}$ converges uniformly when $\mathrm{Re}(s)>1$.
Since the uniform limit of holomorphic functions is holomorphic, the previous step implies that $\log(\zeta(s))$ is holomorphic for $\mathrm{Re}(s)>1$. Then you can differentiate the series of $\log(s)$ term by term to obtain $$ \frac{\zeta'(s)}{\zeta(s)}=\frac{d}{ds} \log(\zeta(s))=\sum_{m,p} \frac{d}{ds}\frac{p^{-ms}}{m}=-\sum_{m,p} (\log p) p^{-ms}.$$ The equality $-\sum_{m,p} (\log p) p^{-ms}=-\sum_{n=1}^{\infty} \frac{\Lambda(n)}{n^{s}}$ comes from the definition of the Von Mangoldt function: $$\Lambda(n)=\begin{cases} \log p & \text{if }n=p^{m}, \\ 0 & \text{otherwise} \end{cases}.$$ Thus $$\sum_{n=1}^{\infty} \frac{\Lambda(n)}{n^{s}}=\sum_{\substack{p \text{ prime} \\ m \geq 1}} \frac{\log p}{p^{ms}}.$$
I will show that $\sum_{n=1}^{\infty} \frac{\Lambda(n)}{n^{s}}$ converges uniformly using the Weiestrass $M$-test. As before, take $s$ with $\mathrm{Re}(s)>1$, and let $c>1$ such that $\mathrm{Re}(s)>c$. By definition of the Von Mangoldt function you have the following bound
\begin{align*} \left| \sum_{n=1}^{\infty} \frac{\Lambda(n)}{n^{s}} \right| &\leq \sum_{n=1}^{\infty} \left| \frac{\Lambda(n)}{n^{s}} \right| \\ & =\sum_{n=1}^{\infty} \frac{\Lambda(n)}{n^{\mathrm{Re}(s)}} \\ & \leq \sum_{n=1}^{\infty} \frac{\log(n)}{n^{c}}. \end{align*}
Since for any $A>0$ we have $\log(n)<\frac{n^{A}}{A}$, we obtain that
$$ \sum_{n=1}^{\infty} \frac{\log(n)}{n^{c}} \leq \frac{1}{A} \sum_{n=1}^{\infty} \frac{1}{n^{c-A}}.$$
If you choose $A$ such that $c-A>1$ (which exists because $c>1$), then you obtain that the RHS of this inequality is finite. So, in virtue of the Weiestrass $M$-test the series of holomorphic functions $\sum_{n=1}^{\infty} \frac{\Lambda(n)}{n^{s}}$ is uniformly convergent. This kind of convergence allow you to change the integral with the series.