In the answer here, Claude Leibovici transforms the integral $$\displaystyle \int_{0}^{1} \frac{3x^3 - x^2 + 2x - 4}{\sqrt{x^2 - 3x + 2}} \, dx$$ with a non-injective substitution $$\sqrt{x^2-3x+2}=t+x\implies x=\frac{2-t^2}{2t+3}\implies dx=-\frac{2 (t+1) (t+2)}{(2 t+3)^2}\,dt$$
At the end, they claim that the bounds of the original integral transform from $[0, 1]$ into $[\sqrt2, -1]$. However, for the lower bound, both $\pm\sqrt2$ are options. How do we know which one to choose(or rather, how did we know choosing the positive one is correct)?
Given that the substitution is $\sqrt{x^2-3x+2}=t+x$,
if $t=-\sqrt{2}$, and since $0<x<1$ this will result in $t+x<0$ i.e. $\sqrt{x^2-3x+2}<0$ so, in this case and for at least a part of the interval of $x$ the substitution is not well defined.
I think your main glitch is trying to substitute for $x$ in $x=\frac{2-t^2}{2t+3}$ and solving for $t$ rather than in the original substitution $\sqrt{x^2-3x+2}=t+x$.