Consider
$$\int_{-2}^1 \frac{\text{d}x}{x} = \left(\lim_{\epsilon \to 0} \int_{-2}^{0-\epsilon} + \int_{0+\epsilon}^1\right) \frac{\text{d}x}{x} = \lim_{\epsilon \to 0} (\ln|-\epsilon| - \ln|-2| + \ln|1| - \ln|0 + \epsilon|$$
And this is
$$\lim_{\epsilon \to 0} \ln|-\epsilon| - \ln(2) - \ln|\epsilon| = \lim_{\epsilon \to 0} \ln\bigg|\frac{-\epsilon}{\epsilon}\bigg| - \ln(2) = -\ln(2)$$
Instead it diverges. Why?
As pointed out in the comments, in order for this integral to converge we require separate limits to converge, i.e. $$ \int_{-2}^1 \frac{\text{d}x}{x} = \left(\lim_{\epsilon \to 0^+} \int_{-2}^{-\epsilon} + \lim_{\delta \to 0^+}\int_{\delta}^1\right) \frac{\text{d}x}{x}. $$ Since the individual limits to not converge the integral diverges.
What you have discovered is an example of the Cauchy Principal Value (CPV), which is a technique of singular integral regularization, in which a finite value can be assigned to certain divergent integrals.
We note that the CPV is just one way we can regularize this integral since $$ \int_{-2}^1 \frac{\text{d}x}{x} ``=" \left(\lim_{\epsilon \to 0^+} \int_{-2}^{-3\epsilon} + \int_{\epsilon}^1\right) \frac{\text{d}x}{x} = \log(3/2) $$ also provides a regularized value that is different than the CPV. What makes the CPV seem intuitively correct is that it assigns a value of zero to divergent integrals of odd functions on a symmetric interval of integration. For example $$ \mathrm{p.v.}\int_{-1}^1\frac{\mathrm dx}{x}=0, $$ which seems like the only sensible value you could assign to this integral.
Furthermore, the CPV is just a special case of a family of singular integral regularizations. For poles of $n=2$ one may use Hadamard regularization and in general, integrands containing poles of arbitrary integer order can be regularized via the finite part integral described by Charles Fox here.
Edit:
The Analytic Principal Value is another interpretation of Fox's regularization which amounts to integrating around a pole by deforming the path of integration into the upper/lower complex plane.