I have to find the solutions of the equation,
$$\tan^{-1}\left(\frac{2x}{1-x^2}\right)+\cot^{-1}\left(\frac{1-x^2}{2x}\right)=\frac{2\pi}{3}$$
So I know that \begin{align}\tan^{-1}\left(\frac{2x}{1-x^2}\right) &= 2\tan^{-1}x\\ \cot^{-1} y &=\tan^{-1}\frac{1}{y}\end{align}
Using these I found the solution to be $$x = \frac{1}{\sqrt3}$$
But my book says that $-\sqrt3$, $\sqrt3 +2$, $\sqrt3 -2$ are also solutions.
How do I obtain those?

Before I start solving, I would like to say that the formula you mentioned:
$$\cot^{-1}(x) =\tan^{-1}\left(\frac{1}{x}\right)$$
is not always true. You can verify this by graphing $\cot^{-1}(x)$ and $\tan^{-1}\left(\frac{1}{x}\right)$ on Desmos. This is only correct if $x > 0$.
Here is the correct statement (from Wikipedia):
So to solve the problem, let's consider 2 cases:
Case $1$: $\frac{1-x^2}{2x} > 0 \text{ (1)}$
$$\tan^{-1}\left(\frac{2x}{1-x^2}\right)+\cot^{-1}\left(\frac{1-x^2}{2x}\right)=\frac{2\pi}{3}$$ $$ \implies \tan^{-1}\left(\frac{2x}{1-x^2}\right)+\tan^{-1}\left(\frac{2x}{1-x^2}\right)=\frac{2\pi}{3}$$ $$ \implies 2\tan^{-1}\left(\frac{2x}{1-x^2}\right)=\frac{2\pi}{3}$$ $$ \implies \tan^{-1}\left(\frac{2x}{1-x^2}\right)=\frac{\pi}{3}$$ $$ \implies \frac{2x}{1-x^2}= \tan\left(\frac{\pi}{3}\right)$$ $$ \implies \frac{2x}{1-x^2}= \sqrt{3}$$ $$ \implies \sqrt{3} \text{ } x^2 + 2x - \sqrt{3} = 0$$ $$ \implies x = -\sqrt{3} \text{ or } x = \frac{1}{\sqrt{3}} \text{ (Both satisfy (1)) }$$
Case $2$: $\frac{1-x^2}{2x} < 0 \text{ (2) }$
$$\tan^{-1}\left(\frac{2x}{1-x^2}\right)+\cot^{-1}\left(\frac{1-x^2}{2x}\right)=\frac{2\pi}{3}$$ $$\tan^{-1}\left(\frac{2x}{1-x^2}\right)+\cot^{-1}\left(\frac{1-x^2}{2x}\right) - \pi =\frac{2\pi}{3} - \pi$$ $$ \implies \tan^{-1}\left(\frac{2x}{1-x^2}\right)+\tan^{-1}\left(\frac{2x}{1-x^2}\right)=\frac{-\pi}{3}$$ $$ \implies 2\tan^{-1}\left(\frac{2x}{1-x^2}\right)=\frac{-\pi}{3}$$ $$ \implies \tan^{-1}\left(\frac{2x}{1-x^2}\right)=\frac{-\pi}{6}$$ $$ \implies \frac{2x}{1-x^2}= \tan\left(-\frac{\pi}{6}\right)$$ $$ \implies \frac{2x}{1-x^2}= -\frac{1}{\sqrt{3}}$$ $$ \implies x^2 - 2\sqrt{3} \text{ }x - 1 = 0$$ $$ \implies x = \sqrt{3} \pm 2 \text{ (Both satisfy (2)) }$$