To find the $\lim\limits_{x\to \infty}\sqrt{x^2+x}-x$ by using binomial expansion, we would first factor out $\sqrt {x^2}=x$ to make expression in the form $(1+p)^n$ like below:
$$\lim\limits_{x\to \infty} \sqrt{x^2+x}-x = \lim\limits_{x\to \infty} x \left({(1+\frac1x)^{\frac12}}-1\right)=\lim _{x\to\infty} x \left(1+\frac12.\frac1x+O(\frac1{x^2})-1\right)=\frac12+O(\frac1x)=\frac12$$
But I'm wondering can we factor out $\sqrt x$ instead, to find the limit? $$\lim\limits_{x\to \infty} \sqrt{x^2+x}-x=\lim_{x\to\infty}\sqrt x\left((1+x)^{\frac12}-\sqrt x\right)=\lim_{x \to \infty} \sqrt x\left(1+\frac12x+\frac{\frac12\times\frac{-1}2}{2!} x^2+O(x^3)-\sqrt x\right)$$ It really looks wrong. But I can't find the mistake. Also is it possible to proceed by factoring out $\sqrt x$ or only taking out $\sqrt {x^2}$ works? (I'm asking because doing either of them makes $(1+p)^n$ form).
I do not believe it is possible to get to a solution from the last line of your evaluation.
The expression $(1+x)^{1/2}$ is approximately $\sqrt x + \frac{1}{2\sqrt x}$ for large $x,$ but $1 + \frac12 x - \frac18 x^2$ is not close at all--it even has the wrong sign. The terms that you have represented by "$O(x^3)$" are needed to correct the value, but you have lost the necessary details to evaluate the series.
Even summarizing all these terms as $O(x^3)$ is incorrect in this context, since as $x$ goes to infinity the $x^4$ term will dominate the $x^3$ term, the $x^5$ term will dominate $x^4,$ and so forth.