Let $\sigma$ be a non-polynomial continuous activation function. The classical universal approximation theorem says that functions of the form $f(x) = C\sigma(Ax + b)$ can approximate any continuous function from a compact subset of $\mathbb{R}^d$ to $\mathbb{R}^D$.
Is there a similar class of injections that can approximate any continuous injection from a compact subset of $\mathbb{R}^d$ to $\mathbb{R}^D$? For example, would it suffice to constrain $\sigma$ in some way and constrain $CA$ to be injective? If so, how could the latter constraint be achieved in a way that allows all parameters to be optimized through gradient descent, while retaining universality?
A related question was asked many years ago, but did not receive an answer.