Given a function $f: X \to Y$, not necessarily surjective, is there a common name (and more concise definition than follows) for a function which maps elements in $Y$ where $f$ is defined to elements of their preimage. In other words, let $Y'$ be the image of $X$ under $f$, the function $h$ should for all $y' \in Y'$ satisfy $f(h(y')) = y'$.
If $f$ is surjective this is called the right-inverse, but otherwise?
And for bonus points is there a name for the set of all such functions with respect to $f$
As you implicitly said, the term "inverse function" only makes sense when talking about bijective functions. The question is if there are generalizations of the concept "inverse function" to functions that are not bijective.
Two important generalizations exist:
(1) A left-inverse function (also called retraction) is a function $g$ that fulfills $g\circ f=\mathrm{id}_X$.
(2) A right-inverse function (also called co-retraction) is a function $h$ that fulfills $f\circ h=\mathrm{id}_Y$.
So the term you are looking for is "left-inverse function".
Please note: Left-inverse functions exist if and only if $f$ is injective. Right-inverse functions exist if and only if $f$ is surjective.