We write $SL_2(\mathbb{Z})$ as $\coprod\alpha_i\Gamma_0(4)$, where $\coprod$ means the disjoint union, and the $\alpha_i$ are the coset representatives of $SL_2(\mathbb{Z})\diagup\Gamma_0(4)$.
We were told that the cusps are then the points $\alpha_i\infty$, as $\alpha_i$ varies.
I understand this, but then I am having trouble working out what the coset reps are for this particular example.
Since $$\Gamma_0(4)=\left\lbrace \begin{pmatrix} a&b\\4c&d\\ \end{pmatrix}:a,b,c,d\in \mathbb{Z}\right\rbrace $$
I thought the representatives could be something like $\left\lbrace\begin{pmatrix}a&b\\c'&d\\ \end{pmatrix}\right\rbrace$, where $0\leq c'\leq 3$ and $a,b,d$ are arbitrary.
But then I know that $SL_2(\mathbb{Z})\diagup\Gamma_0(4)$ has order $6$, due to the formula $\left(SL_2(\mathbb{Z}):\Gamma_0(N)\right)=N\prod_{p|N} (1+\frac 1p)$.
This may be more of an algebraic problem than a modular forms one specifically, but I am struggling here!
Let me begin with some generalities:
Let $R$ be a ring with identity and $\mathbf{P}^1(R)$ the projective line over $R$. To define this gadget, consider the equivalence relation $\sim$ on $R \times R$ defined by $(a, b) \sim (c, d)$ if there is a unit $u$ such that $ua = c$ and $ub = d$. Note now that the ideal $aR + bR$ generated by $(a, b)$ depends only on the equivalence class of $(a, b)$. Put $\mathbf{P}^1(R)$ for the set of all equivalence classes for which this ideal generated is the full ring $R$.
In the case of the ring $\mathbf{Z}/N\mathbf{Z}$, there is the following fact:
This fact allows us to choose representatives for our class $(c: d)$ so that $(c, d) = 1$.
Now, we have the following general proposition:
In this case, we have
$$\mathbf{P}^1(\mathbf{Z}/4\mathbf{Z}) = \{(0:1), (1:0), (1:1), (1:2), (1:3), (2:1)\} $$ and a set of coset representatives for $\Gamma_0(4)$ in $SL_2(\mathbf{Z})$ is: $$\left\{ \begin{pmatrix}1 &0 \\ 0 & 1 \end{pmatrix}, \begin{pmatrix} 0 & -1 \\ 1& 0\end{pmatrix}, \begin{pmatrix} 1 & 0 \\ 1 & 1\end{pmatrix}, \begin{pmatrix} 0 & -1 \\ 1 & 2\end{pmatrix}, \begin{pmatrix} 0 & -1 \\ 1 & 3\end{pmatrix}, \begin{pmatrix}1 & 0 \\ 2 & 1 \end{pmatrix}\right\}.$$
Edit. Indeed, there are theorems that give coset representatives for other important congruence subgroups (viz. $\Gamma_1(N)$ and $\Gamma(N)$). See L. Kilford's book starting from Proposition 2.11. He discusses this at length. (Let me add that you'll find the following group theory fact useful: if $K \subseteq H \subseteq G$ are groups, then, there is an explicit bijection between $[G: K]$ and $[G: H] \times [H: K]$.)