Basic Question about Principal Congruence Subgroups

Question

I'm using Diamond and Shurman's text on Modular Forms and the section on congruence groups states that there is a natural homomorphism from $SL(2,\mathbb{Z}) \rightarrow SL(2, \mathbb{Z}/N\mathbb{Z})$ given by reducing entrywise modulo $N$. My question is why is the image of this map necessarily in $SL(2,\mathbb{Z}/N\mathbb{Z})$? For example, for $N = 4$, if we took $\begin{pmatrix} 3 & 4\\ 2 & 3\\ \end{pmatrix}$ and reduced, we would get $\begin{pmatrix} 3 & 0\\ 2 & 3\\ \end{pmatrix}$ which has determinant 9. Perhaps I'm confused on the definition of $SL(2, \mathbb{Z}/N\mathbb{Z})$. Do we reduce the determinant modulo 4 as well?