Let $p(x) = \prod_{i} \left( x - \lambda_i \right)^{m_i}$ be a polynomial where the roots $\lambda_i$ may be complex but $p(x)$ has real coefficients (so complex roots must come in conjugate pairs).
It is know that the logarithmic derivative of $p(x)$ has the form
$$ \frac{p'(x)}{p(x)} = \sum_{i} \frac{m_i}{x - \lambda_i}. $$
Let $\gamma$ be a closed contour in the complex plane that contains the complex root $\lambda_1$ and does not contain any other root. Applying the residue theorem then we have
$$ \frac{1}{2 \pi i}\oint_\gamma \frac{p'(x)}{p(x)} dx = m_1. $$
This is because $m_1$ is the residue of $\frac{p'(x)}{p(x)}$ at $\lambda_1$. Indeed:
$$ \lim_{x\to\lambda_1} \left(x - \lambda_1 \right) \frac{p'(x)}{p(x)} = m_1. $$
First question: is my understanding up to here correct?
On the other hand, I am trying to solve this integral by hand as follows. Parameterize $\gamma$ as $x(t) = \lambda_1 + \epsilon e^{i t}$ for $t \in [0, 2\pi]$m, where $\epsilon$ is small enough so that $\gamma$ only contains $\lambda_1$ and no other root. Then we have
$$ \oint_\gamma \frac{p'(x)}{p(x)} dx = \int_0^{2 \pi} \frac{p'(\lambda_1 + \epsilon e^{i \pi t})}{p(\lambda_1 + \epsilon e^{i t})} (\epsilon i e^{i t}) dt = \ln ( p(\lambda_1 + \epsilon e^{i 2 \pi})) - \ln ( p(\lambda_1 + \epsilon e^{i 0})) = \ln (p(\lambda_1 + \epsilon )) - \ln (p(\lambda_1 + \epsilon )) = 0. $$
Can anyone please correct this last integral? I am pretty sure the first part of my understanding is correct. What is the problem with the way I'm solving the integral? TIA.
Yes, your understanding up to the point where you get that the residue is $m_1$ is correct.
Concerning the rest, the problem lies in using an undefined function, which is $\ln$. There is no holomorphic function $\ln\colon\Bbb C\setminus\{0\}\longrightarrow\Bbb C$ such that $(\forall z\in\Bbb C\setminus\{0\}):\ln'(z)=\frac1z$. If there was, we would have $\int_{|z|=1}\frac1z\,\mathrm dz=0$, by the same argument as yours, but, in fact, $\int_{|z|=1}\frac1z\,\mathrm dz=2\pi i$.