Given a group $ G $ (I've only really used Lie groups), I am looking for a way to transform construct a transformation $ T_G: \mathbb{R}^n \rightarrow \mathbb{R}^m $ where $ m = \text{dim}(\mathbb{R}^n/G) $ such that,
\begin{align} T_G(x) &= T_G(U_g x)~~\forall g \in G \\ T_G(x) &\ne T_G(y) \iff U_g x \ne y~~ \forall g \in G \end{align} where $ U_g $ is a representation for the group element.
For example, if $ G = SO(2) $, then $ T_G $ would "remove" the angle coordinate $ \phi $ and give $ T_G(x) \propto \sqrt{x_1^2 + x_2^2} $ because $ T_G(U_g x) = \| U_g x \| = \| x \| = T_G(x) $ and iff $ \| x \| \ne \| y \| $ then $ T_G(x) \ne T_G(y) $.
EDIT:
See anon's comment. He worded it more properly.
My guess is that by a "transformation" $T_G$ you mean a continuous map which projects to a continuous injective map $R^n/G\to R^m$. Easy examples come from non-Hausdorff quotient spaces: $G={\mathbb R}^\times$, $n=1$. Then $R^1/G$ is a 2-point set with non-Hausdorff topology; hence, it admits only constant continuous maps to $R^k$ no matter what $k$ is.
OK, maybe you will want to restrict to compact subgroups $G$. Consider then the example $G\cong {\mathbb Z}/2$, whose generator acts on $R^3$ as the antipodal map. The quotient $R^3/G$ is the cone over the projective plane, hence, does not embed in $R^3$. Similar examples exist of you insist on compact connected Lie group: Take $U(1)$ acting on $C^3$. The quotient is a cone over $CP^2$, it does not embed in $R^5$.