According to the definition of the cusps attached above the set of cusps is infinite and to be more precise it is $Q \cup \infty$ since $G$ is a subgroup of $SL(2,Z)$ so the identity matrix (that is a parabolic element) belongs to $G$ so every point of $Q \cup \infty$ is cusp but they mentioned before that it is finite.
How could this be explained? (In addition, what is cusp?)
$SL_2(\Bbb{Z}) i\infty = \Bbb{Q}\cup i\infty$
$SL_2(\Bbb{Z}) i\infty $ is only one point of the modular curve, the one that you need to add to get a compact Riemann surface. The chart is $q\to SL_2(\Bbb{Z}) \frac{\log(q)}{2i\pi}$ for $|q|<1/3$
It works the same way for $\Bbb{Q}\cup i\infty=\bigcup_{j=1}^J G \gamma_j i\infty$ a finite disjoint union, for some $\gamma_j \in SL_2(\Bbb{Z})$, and we need to add $J$ points to $G\backslash \Bbb{H}$ to get a compact Riemann surface. The charts are $q\to G \gamma_j k_j\frac{\log(q)}{2i\pi }$ for some $k_j$.
This is what we mean with the cusps of the modular curves.