Define Klein's modular $j$-invariant as $$j(\tau) = 1728 \frac{g_2^3}{\Delta},$$ where $$ g_2 = 60 \sum_{(m,n)}' \Big( \frac{1}{m + n \tau} \Big)^4$$ and $$ g_3 = 60 \sum_{(m,n)}' \Big( \frac{1}{m + n \tau} \Big)^6$$ and $$\Delta = g_2^3 - 27 g_3^2.$$
It is well known (see e.g. Apostol Modular Forms, Theorem 1.20) that the Fourier expansion of $j(\tau)$ has integral coefficients, starting with $$j(\tau) = q^{-1} + 744 + 196883 q + ...$$
It is also well known that $j(\tau)$ is the principal modulus or hauptmodul for $\mathrm{SL}_2(\mathbb{Z})$, meaning that it is the unique generator for the field of functions on $X(\mathrm{SL}_2(\mathbb{Z}))$ and serves as the isomorphism between $X(\mathrm{SL}_2(\mathbb{Z}))$ and the Riemann sphere. Every other genus zero subgroup in $\mathrm{SL}_2(\mathbb{R})$ has a principal modulus.
My question (finally) is: Do the Fourier expansions of hauptmoduln all have integral coefficients? I have looked up a couple (namely the hauptmoduln for $\Gamma_0(2)$ and $\Gamma_0(3)$ and this appears to be the case, but I am not sure how to look this up more generally.
Moreover, if the answer to my first question is yes, I want to ask: Is there a conceptual reasons for why the fourier expansions of hauptmoduln are integral? I ask the second question in particular because Apostol does not even introduce the notion of a hauptmodul, and his proof that the Fourier expansion of $j(\tau)$ has integral coefficients comes by manipulating the Eisenstein series given above.