$$z^2-2(1-2i)z-8i=0$$ Here is my working: $$\begin{align} x_{1,2} &= \frac{2-4i \pm \sqrt{(4i-2)^2 + 4\cdot 8i}}{2} \\ &=\frac{2-4i \pm \sqrt{-16+4-16i + 32i}}{2} \\ &= \frac{2-4i \pm \sqrt{-12+16i}}{2} \\ &= \frac{2-4i \pm \sqrt{4(-3+4i)}}{2} \\ &= \frac{2-4i \pm 2\sqrt{(4i-3)}}{2} \\ &= 1-2i \pm \sqrt{4i-3} \end{align}$$ Let $a+bi = \sqrt{4i-3}$
$$a^2-b^2+abi=4i-3$$
$$a^2-b^2=-3, ab=4$$
$$b=\frac4a$$
$$a^2-\left(\frac{4}{a}\right)^2=-3$$
$$a^4+3a^2-16=0$$
$$t^2+3t-16=0$$
$$t_{1,2}=\frac{-3\pm\sqrt{9+4\cdot 16}}{2}$$
Answers should be $z=2, z=-4i,$ I don't know what I am doing wrong, I can get it right when substituting $z=a+bi$ but I don't when using this method.
The mistake is when you square $a+bi$, you should get $a^2-b^2+\color{red}2abi$ and not $a^2-b^2+abi$.
$$a^4+3a^2-4=0$$ $$(a^2+4)(a^2-1)=0$$ $$a=\pm 1, b = \pm 2$$ Hence, $$x_{1,2}=1-2i\pm(1+2i)$$
which gives you the desired solution of $2$ and $-4i$.