Is there any obstruction other than Riemann-Hurwitz to the existence of covers of Riemann surfaces?

Question

Suppose $X$ is a genus $g$ Riemann surface, and $h,d,e_i$ are positive integers such that $2-2g = d(2-2h) + \sum (e_i-1)$. Is there necessarily a Riemann surface $Y$ with a map $f: Y \rightarrow X$ such that $f$ has degree $d$, $Y$ has genus $h$, and the ramification numbers are precisely the $e_i$? (Refinement: can we choose whatever branch points we want on $X$?)