Question: Consider a $L$-shaped path $L_\epsilon:\frac{1}{2}+\epsilon\to \frac{1}{2}+\epsilon+i\ (H+\epsilon)\to \frac{1}{2}+i\ (H+\epsilon)$ where $H>0$ is fixed and $\epsilon>0$ is arbitrarily small. If the Riemann zeta function, $\zeta(s)$ is non zero on $L_\epsilon$ then prove that $$\int_{L_\epsilon} \frac{d}{ds}\log[(s-1)\zeta(s)]\ ds = \log\left[\left(-\frac{1}{2}+i(H+\epsilon)\right)\zeta\left(\frac{1}{2}+i(H+\epsilon)\right)\right]-\log\left(\left(\epsilon-\frac{1}{2}\right)\zeta\left(\frac{1}{2}+\epsilon\right)\right)$$ My Try: Since the function $f(s)=(s-1)\zeta(s)$ is entire and $f(s)$ has no zeros on $L_\epsilon$ so we can for each point $s'\in L_\epsilon$ find a small neighbourhood $U(s')$ so that $f(s)\neq 0$ $\forall s\in U(s')$. Hence we can find an open set $U=\bigcup\limits_{s'\in L_\epsilon} U(s') $, so that $L_\epsilon\subset U$ and $f(s)\neq 0$ for every $s\in U$.
But Fundamental Theorem of Calculus would not work due to singularities involved and the logarithm of the zeta function must be defined carefully.
If $f$ is a holomorphic function on a simply connected region $U$ and $f(z) \neq 0$ for all $z\in U$, there exists a holomorphic function $g$ such that $\exp \circ g = f$. This is a standard result in complex analysis. See the reference cited here.
You have constructured the simply connected region $U$ on which $f(s) = (s-1)\zeta(s)$ is non-vanishing and holomorphic so just apply this result. Then you can use the fundamental theorem of calculus to conclude.