Let $H$ be the Hilbert scheme parametrizes subschemes of $\mathbb P^{5g-6}$ with Hilbert polynomial $p(m)=(6g-6)m+(1-g)$ (for example, curves with genus $g$ embedding by the canonical bundle to the 3rd power), and let $\pi: U \to H$ be the universal family. Let $\omega=\omega_{U/H}$ be the relative dualizing sheaf. I want to know that
how to compute the self intersection number $det(\pi_* \omega)^n$? Where $n$ is the dimension of $H$.
Although this is may not be well defined on every point, those bad ones are of higher codimension, so I believe this number is still well defined. Could anyone give a comment or reference on this question? Thanks in advance.