Classically, the Hecke Operators act as endomorphisms of $\omega^k_{\mathcal{M}_{ell}(\mathbb{C})}$, defined by noting that there is a distinguished class of covering maps $\widetilde{E}\to E$ given by noting that $E=\mathbb{C}/\Lambda$, and $\Lambda$ has a finite number of sub-lattices of degree $n$. We have that the $\widetilde{E}$ are elliptic curve, so summing over the form on these curves, we obtain the Hecke Operators. My question is whether this can be extended to more general varieties?
For example, let $X$ be a smooth complex variety. An elliptic curve $E\to X$ one should have that etale covers $\widetilde{E}\to E\to X$ of degree $n$ have $\widetilde{E}$ elliptic, so we should be able to sum over some space of etale covers, but I don't know enough about how to make this work.