References about moduli space of abelian varieties with level structure

Question

In the course of one of my research project, I have been advised to try to have a look to "Moduli Space of Abelian Varieties with Level Structure".

I am interested in references where this topic is threated in some detail(I already found some material that i am digesting, but i would like to see if there is some very standard reference that i may be missing and that would save me time)

p.s:Indeed what i need are technique to realize abstract representation of a group, as a Galois action on the torsion of an abelian variety(I have a very explicit action, given by matrices). The first dimension that i would need(of the Abelian variety) is 3. But I have a family of representations that i would like to realize of rank going to infinity. This to say that I am interested to a theory for arbitrary dimension, or at least that works in dimension 3.

Answer 1

I've made this community wiki for obvious reasons.

1. Arithmetic Moduli of Elliptic Curves by Katz and Mazur. This is the ideal comprehensive reference if you want to work with just elliptic curves. It has pretty much everything known at the time (and much more—a lot of the work in their is original). But, it's super long, and super detailed. So, it can be intimidating on a first go. I included this, even though you said dimension at least 3, since it often times provides a more concrete realization of the higher-dimensional theory.
2. Geometric Invariant Theory by David Mumford. This contains the proof that the moduli space polarized abelian varieties (of sufficiently large degree) with level structure is representable. It's also dense, especially considering it's contained within a huge book a lot of the material of which you might not want to learn.
3. $p$-adic Automorphic Forms and Shimura Varieties by Haruzo Hida. This sounds super scary, but the first two sections of Chapter 4 and all of Chapter 6 contain a nice discussion of moduli spaces of abelian varieties with level data which is self-contained (almost to a fault).
4. Points on Some Shimura Varieties over Finite Fields by Robert Kottwitz. This is not entirely relevant to you, but if you want to understand some larger classes of moduli problems (the key word being 'PEL type Shimura variety') then this contains a discussion of their representability, and basic properties.
5. Arithmetic Moduli of Generalized Elliptic Curves by Brian Conrad. Again, I know you said dimension at least 3, but this gives a great discussion of the theory for elliptic curves from a stack-theoretic point of view.
6. Complex Abelian Varieties by Birkehnake and Lange. This is a pretty definitive source on the complex theory. It sounds like you're interested in the arithmetic theory, but step one is understanding the geometric theory.