I am wondering if there is a systematic way to find the indefinite integral
$$\int \left(\frac{\arctan x}{\arctan x - x}\right)^2 dx.$$
It indeed has a clean closed form $$\frac{x^2 + 1}{\arctan x - x} + x.$$
But I am not able to reach it in a logical way. The transformation $t = \arctan x$ doesn't seem to help much.
On
To find the indefinite integral I will make use of the so-called reverse quotient rule (For another example using this method see here).
Recall that if $u$ and $v$ are differentiable functions, from the quotient rule $$\left (\frac{u}{v} \right )' = \frac{u' v - v' u}{v^2},$$ it is immediate that $$\int \frac{u' v - v' u}{v^2} \, dx = \int \left (\frac{u}{v} \right )' \, dx = \frac{u}{v} + C. \tag1$$
Writing the integral as $$I = \int \left (\frac{\arctan (x)}{\arctan (x) - x} \right )^2 \, dx = \int \frac{\arctan^2 (x)}{(\arctan (x) - x)^2} \, dx,$$ we see that $v = \arctan (x) - x$. So $v' = -x^2/(1 + x^2)$. Now for the hard bit. We need to find a function $u(x)$ such that $$u' v - v' u = u'(\arctan (x) - x) + u \frac{x^2}{1 + x^2} = \arctan^2 (x).$$ After a little trial and error we find that if $$u = x\arctan (x) + 1,$$ as $$u' = \arctan (x) + \frac{x}{1 + x^2},$$ then $$u' v - v' u = \arctan^2 (x),$$ as required.
Our integral can now be readily found as it can be rewritten in the form given by (1). The result is: $$I = \int \left (\frac{x \arctan (x) + 1}{\arctan (x) - x} \right )' \, dx = \frac{x \arctan (x) + 1}{\arctan (x) - x} + C.$$
The integrand is $$ \left(1+\frac{x}{\arctan x - x}\right)^2 = 1 + \frac{2x}{\arctan x - x} + \frac{x^2}{(\arctan x - x)^2} $$
Observe that $$ (\arctan x - x)' = \frac{1}{x^2+1} - 1 = -\frac{x^2}{x^2+1} $$
So we can perform IBP on the last term
$$ \int (x^2+1)\left(\frac{1}{(\arctan x - x)^2}\frac{x^2}{x^2+1}\right)dx = \frac{x^2+1}{\arctan x - x} - \int\frac{2x}{\arctan x - x} dx $$
Thus $$ \int \frac{x^2}{(\arctan x - x)^2}dx + \int\frac{2x}{(\arctan x - x)^2} dx = \frac{x^2 + 1}{\arctan x - x} $$
The rest is obvious.