I solved it by rewriting the limit as indeterminate form $0/0$, then apply L'Hopital's rule 4 times, It was really lengthy and easy to make mistakes, If anyone got a better approach, please tell me!
$$ \begin{align} \lim_{x\to0^+}(\cot x-\frac{1}{x})(\cot x+\frac{1}{x})&=\lim_{x\to0^+}\frac{x^2\cos^2x-\sin^2x}{x^2\sin^2 x}\\ \vdots\\ &=\lim_{x\to0^+}\frac{-16\cos 2x+\ldots}{24\cos 2x+\ldots}\\ &=-\frac{2}{3} \end{align} $$
Using $$\tan(x)=x+\frac{x^3}{3}+o(x^3)\quad \text{and}\quad \frac{1}{1+x}=1-x+o(x),$$ yields \begin{align*} \left(\cot(x)-\frac{1}{x}\right)\left(\cot(x)+\frac{1}{x}\right)&=\cot^2(x)-\frac{1}{x^2}\\ &=\frac{1}{\tan^2(x)}-\frac{1}{x^2}\\ &=\frac{1}{x^2+\frac{2}{3}x^4+o(x^4)}-\frac{1}{x^2}\\ &=\frac{1}{x^2}\left(\frac{1}{1+\frac{2}{3}x^2+o(x^2)}-1\right)\\ &=\frac{1}{x^2}\left(1-\frac{2}{3}x^2+1+o(x^2)\right)\\ &=-\frac{2}{3}+o(1). \end{align*}