In many books, I see people defining modular forms to be holomorphic/meromorphic functions in the upper half plane such that it is invariant under the $|_k$ action of the group $\operatorname{SL}_2(\mathbb{Z})$ or under a congruence subgroup of it ($f|_k A=f \quad \forall A\in \operatorname{SL}_2(\mathbb{Z})$ ).
In the same time, they define the slash operator $|_k$ for bigger groups like $\operatorname{SL}_2(\mathbb{R})$ or $\operatorname{SL}_2(\mathbb{Q})$, but I never saw anyone define modular form to be invariant under the $|_k$ action of $\operatorname{SL}_2(\mathbb{R})$ or $\operatorname{SL}_2(\mathbb{Q})$
My question: why $\operatorname{SL}_2(\mathbb{Z})$ and its subgroups are always used to define modular forms, and why we do not define them usually for bigger groups?
Thanks.
Mike Miller's response is clear and cuts to the chase. If you are still somewhat unsatisfied, to help build intuition consider the following observations:
Defining the Modular group as $SL_2(\mathbb{Z})$ allows us to make the claim that if $f$ is a weakly modular function, then $f$ is periodic on $\mathbb{Z}$. Indeed since $\alpha = \left( \begin{array}{cc} 0 & 1 \\ 1 & 1 \end{array} \right) \in SL_2(\mathbb{Z})$ we have that $f(z + 1) = (0(z) + 1)^{k}f(z) = f(z)$.
Note that this is not a unique property of $SL_2(\mathbb{Z})$ since $\alpha$ is also an element of $SL_2(\mathbb{R})$ but $SL_2(\mathbb{Z})$ is in a loose sense the "smallest" group that has this property. But you should still be upset at me since this I should more appropriately and accurately call the action of $SL_2(\mathbb{Z})$ on modular forms in general (not just weakly modular functions) as being "integrally periodic", i.e. the symmetry involved in the periodicity of modular forms is somewhat related with cyclotomy, for which it comes to no surprise that the more modern approaches to the study of modular forms have been along the lines of Galois representations. But I should not talk more about this since it is beyond my scope at the moment.
The most important reason though that the modular group is $SL_2(\mathbb{Z})$, denote by $G$, is that when we are investigating forms which are invariant under a particular subgroup of $G$, we most notably look at congruence subroups such as $$\Gamma_0(N) = \{\left( \begin{array}{cc} a & b \\ c & d \end{array} \right) \in SL_2(\mathbb{Z}) \mid c \equiv 0 \pmod N\}$$ and $$\Gamma_1(N) = \{\left( \begin{array}{cc} a & b \\ c & d \end{array} \right) \in SL_2(\mathbb{Z}) \mid a \equiv d \equiv 1 \pmod N, c \equiv 0 \pmod N\}$$ Now one thing to notice here is that since these are indeed subgroups of $SL_2(\mathbb{Z})$ the elements of these groups have determinant equal to $1$. In particular, when we look at $\Gamma_1$, we could simply have made the definition exclude the condition that $a \equiv 1 \pmod N$ since if $d \equiv 1 \pmod N$ then $a$ must in fact be congruent to $1$ modulo $N$ since the entries are integral. This trend of "forcing" entries to be certain values occurs frequently, especially when we try to understand things like what is a complete set of coset representatives, $\{\beta_j\}$, for $\Gamma_1 \alpha \Gamma_1 = \cup_j \Gamma_1 \beta_j$ and $\alpha = \{\left( \begin{array}{cc} 1 & 0 \\ 0 & p \end{array} \right)$, $p$ a prime.
One thing you may have already noticed is that the notion of a congruence subgroup doesn't make sense if the entries are not integral. As to why we study congruence subgroups in the first place is a question that spans centuries of Number Theory. If you wish to learn more, Diamond and Shurman's A First Course in Modular Forms does a good job of exploring the interplay between Modular Forms and Elliptic curves from an advanced standpoint.