My question is exactly that on the title.
I'm interested in the action of some (discrete) subgroup $H$ on $\operatorname{GL}_n(\mathbb{R})$ by left multiplication.
For example, $H$ can be
- $\operatorname{GL}_n(\mathbb{Z})$ or $\operatorname{PSL}_n(\mathbb{Z})$,
- the set of $n\times n$ diagonal matrices that contain only $1$ or $-1$ on the diagonal.
I want to know the conditions on $H$ that guarantee the compactness of the quotient $\operatorname{GL}_n(\mathbb{R})/H$.
This seems like a well-studied subject, but couldn't find exactly what I want on this Wikipedia page. References are also welcome.
First of all, $GL(n, {\mathbb Z})$ contains $SL(n, {\mathbb Z})$ as index 2 subgroup. Therefore, for $G= GL(n, {\mathbb R})$, $GL(n, {\mathbb Z})\backslash G$ is compact if and only if $SL(n, {\mathbb Z})\backslash G$ is compact. To see that the latter quotient is noncompact, observe that $G$ splits as the direct product $$ SL(n, {\mathbb R})\times {\mathbb R}^\times. $$ This product decomposition is preserved by the left translations via elements of $SL(n, {\mathbb Z})$. Therefore, $SL(n, {\mathbb Z})\backslash G$ is homeomorphic to $$ (SL(n, {\mathbb Z}) \backslash SL(n, {\mathbb R}))\times {\mathbb R}^\times $$ and, hence, is noncompact.
More interestingly, already the quotient $$ SL(n, {\mathbb Z}) \backslash SL(n, {\mathbb R}) $$ is noncompact. This can be seeing as a consequence of the Godement's criterion:
(a) Suppose that $G$ is a semisimple Lie group and $\Gamma< G$ is a discrete subgroup containing unipotent elements. Then $\Gamma\backslash G$ is noncompact.
(b) Suppose that $\Gamma< G$ is a lattice, i.e. $\Gamma\backslash G$ has finite volume (where we equip $G$ with the Haar measure). Then $\Gamma\backslash G$ is noncompact if and only if $\Gamma$ contains nontrivial unipotent elements.
Now, $SL(n, {\mathbb Z})$ contains some nontrivial unipotent elements (strictly upper-triangular matrices). Hence, $$ SL(n, {\mathbb Z}) \backslash SL(n, {\mathbb R}) $$ is noncompact. Incidentally, $SL(n, {\mathbb Z})$ is a lattice in $SL(n, {\mathbb R})$ (since it is an arithmetic subgroup of $SL(n, {\mathbb Q})$).
You can find much more in
Raghunathan, M. S., Discrete subgroups of Lie groups, Ergebnisse der Mathematik und ihrer Grenzgebiete. Band 68. Berlin-Heidelberg-New York: Springer-Verlag. VIII,227 p. (1972). ZBL0254.22005.
Morris, Dave Witte, Introduction to arithmetic groups, [s.l.]: Deductive Press (ISBN 978-0-9865716-0-2/pbk; 978-0-9865716-1-9/hbk). xii, 475 p. (2015). ZBL1319.22007.