I'm trying to find all solutions to $x^2 + 96=0$ in $\mathbb{Z}_{100}$.
$x^2 + 96 \equiv 0 \bmod 100$ implies that $x^2 + 96 \equiv 0 \bmod 2$ and $x^2 + 96 \equiv 0 \bmod 5$.
$$x^2 + 96 \equiv 0 \bmod 2 \iff x^2 \equiv 0 \bmod 2.$$ This means that $x$ must be even, since the square of an odd number is odd. $$x^2 + 96 \equiv 0 \bmod 5 \iff x^2 \equiv 4 \bmod 5 \implies x=\pm2.$$ Is this correct?
You are correct so far. However, rather than considering $\mod 2$ and $\mod 5$ you should consider $\mod 4$ and $\mod 25$, as these modulos are relatively prime and multiply to get $100$.
Mod $4$, you will conclude that $x \equiv 0 \text{ or } 2$. Mod $25$, you can show that $x \equiv \pm 2 \pmod {25}$. Finally, use Chinese Remainder Theorem to find the possible values for $x$ $\mod 100$.