I'm supposed to find:
$$ \int_0^{\infty} \frac{(\ln x)^2}{1+x^2} dx $$
I start of by finding:
$$ I_1 = \oint \frac{(\ln z)^2}{1+z^2} dz $$ $$ = \oint \frac{(\ln z)^2}{(z+i)(z-i)} dz $$
Now I take this semi-circle path:

$$ = 2\pi i \times Residue(z=i) $$ $$= 2\pi i \times \frac{(\ln i)^2}{2i} $$ $$I_1 = -\frac{\pi^3}{4} $$
Now applying cauchy's integral:
$$\int_{-R}^{0} + \int_0^R + \int_\Gamma = -\frac{\pi^3}{4} $$
Taking limits as $R\rightarrow \infty$, $\int_\Gamma \rightarrow 0$
$$\int_{-\infty}^{0} \frac{(\ln x)^2}{1+x^2} + \int_0^\infty \frac{(\ln x)^2}{1+x^2} = -\frac{\pi^3}{4} $$
Here's what I don't get, why did the solution do this:

If $x < 0$, $$\ln (x) = \ln(-|x|) = \ln(e^{i\pi}|x|) = \ln(|x|) + i\pi$$ which is the definition of the complex logarithm for $x < 0$ (for any branch containing $\pi$).
Having said that, your contour is problematic as it goes through zero where $\ln x$ has a singularity. Instead you should consider a contour like half an annulus: a semicircular arc from $R$ to $-R$, then a straight line segment from $-R$ to $-\varepsilon$, then a semicircular arc from $-\varepsilon$ to $\varepsilon$, then finally a straight line segment from $\varepsilon$ to $R$. As $R \to \infty$ and $\varepsilon \to 0$, the integrals along the semicircular arcs go to zero and then you are left with the calculation in your post.