The metric is used ubiquitously in General Relativity. It is a multilinear symmetric tensor $$ g: TM \times TM \to \mathbb{R} $$ which physicists write as $$ ds^2\equiv g=g_{\mu\nu}dx^\mu \ dx^\nu, $$ with a symmetric tensor product implied between the differentials.
The metric is, actually, a section of the appropriate bundle; but let's ignore this complication since I want to direct your attention on the so-called "inverse" metric.
With the extra assumption of $g$ being non-degenerate: given $g$, and a vector $\partial_\mu$, I can define a covector as $$ \left[g(\partial_\mu,\cdot)\right]=g_{\mu\rho} \ dx^\rho $$
In physics they use the inverse metric too, with both indices "upstairs". Viewing it mathematically, I cannot understand the meaning of inverse.
How can I attain something of the sort $$ g^{\mu\nu} $$ given $g_{\mu\nu}$ and demanding that $g^{\mu\nu}g_{\nu\alpha}=\delta^\mu_\alpha$.
Also the word "inverse" seems to be a misnomer; since such thing as $$ g^{-1}: \mathbb{R}\to TM \times TM $$ is unattainable.
The inverse metric is defined by the property that:
$$ g\circ g^{-1} = \delta $$
where the Kronecker delta is defined to be the section of $T^*M\otimes TM$ that is acts as the identity on $T^*M$:
$$ \delta := \sum_{i = 1}^{n} \mathrm{d}x^i \otimes \partial_{x^i} $$
In other words, if there were a general way to compute the inverse metric without computing the inverse matrix, then the method can be applied to the case of a constant metric on a linear space (that is, a simple inner product space) to compute the inverse matrix of a matrix without, computing the inverse matrix.