Suppose we have two finite sets $X$ and $Y$ and a many to one mapping $f:X\rightarrow Y$.
Now let me define another mapping $g:Y\rightarrow\mathcal{P}(X)$ where $\mathcal{P}$ denotes the power set. We define $g(y)=\{x|x\in X, f(x)=y\} \forall y$.
Can we call $g$ as an inverse of $x$ in any meaningful way? I know it is stretching the definition, but is there any terminology for such a mapping?
On
A subset $R$ of the (cartesian) product $A\times B$ of two sets $A$ and $B$ is called a relation from $A$ to $B$. For any relation $R$ we define the opposite $R^{op} = \{(b,a)\,|\,(a,b)\in R\}$ which is a relation from $B$ to $A$. A map is a special kind of relation (i.e a relation $f$ is a map when for each $a$ in $A$ there exists a unique $b$ in $B$ such that $(a,b) \in f$). If $f$ is an injective and surjective map, then its inverse is $f^{op}$. In sets there is a bijection $Rel(A, B) \cong Maps(A, P(B))$ which sends a relation $R$ to the map which sends $a$ to the set $\{b\in B\,|\,(a,b)\in R\}$. Since the map sending a relation $R$ to $R^{op}$ is also a bijection we obtain $Maps(A, P(B))\cong Rel(A,B) \cong Rel(B,A) \cong Maps(B, P(A))$. We could change our perspective at this point and think of a relation as a map $A\to P(B)$, and then its opposite (which as we have seen is something similar to inverse) would be a map $B\to P(A)$. Since every map $A\to B$ determines, by composing with the map $B\to P(B)$ which sends $b$ to $\{b\}$, a map $A\to P(B)$ I would suggest that it makes more sense call what you described as opposite (or something to that effect).
More generally, there is a function $$f^{-1}:\mathcal{P}(Y) \to \mathcal{P}(X):S \mapsto \{x \in X \ \vert \ f(x) \in S\}$$ called the inverse image (or preimage) of $f$.
The map you described is the special case where you only consider singletons. This could be called the fiber map (this is not usual terminology though).