Common ratio = $r=\frac{-1}{2}$
First term=$a$
$$\frac{5}{1024}=5\left(-\frac{1}{2}\right)^{n-1}$$ $$\frac{1}{1024}=\left(-\frac{1}{2}\right)^{n-1}$$ $$1024=(-2)^{1-n}$$ $$(-2)^{10}=(-2)^{1-n}$$ Then $$10=1-n$$ $$n=-9$$ which makes so sense. How should I get the right answer?
It started to be wrong from the third line. You take a $-1$ power both side but you do too much on your right side, it should be: $\left[\left(-\dfrac{1}{2}\right)^{n-1}\right]^{-1}=\left(-2\right)^{n-1}\text{ or }\left(-\dfrac{1}{2}\right)^{1-n}$