I am looking for a (nonlinear) map from $n$-dimensional cube to an $n$-dimensional simplex; to make it simple, assume the following figure which is showing a sample transformation for the case when $n=2$. (sorry for my bad drawing). The two circles in the rectangle, are set of points, which are mapped to the rectangle. Note that, points by this transformation maintain their relative position to each other, as shown in the figure.
http://tinypic.com?ref=30bcxzp
Any suggestion how to find such transformation?
If all you want is a homeomorphism this is pretty simple. I'll do the 2-dimensional case because it's easiest and the leg work for extending to higher dimensions can be annoying if not done carefully.
Let $X=\{(x,y)\in\mathbb{R}^2\:|\:x\in[0,1], y\in[0,1]\}$, let $X'=\{(x,y)\in\mathbb{R}^2\:|\:x\in[0,\frac{1}{2}], y\in[0,\frac{1}{2}]\}$, let $Y=\{(x,y)\in X\:|x+y\leq 1\}\:$. We want a homeomorphism $h\colon X\rightarrow Y$.
Let $g\colon X\rightarrow X'$ be given by $g(x,y)=(\frac{1}{2}x,\frac{1}{2}y)$. Let $f\colon X'\rightarrow Y$ be given by $$f(x,y)=\left\{\begin{array}{lr}(2x-y,y),&\mbox{ if }x\geq y\\ (x,2y-x),&\mbox{ if }x\leq y\end{array}\right.$$ and so both $g$ and $f$ are homeomorphisms and hence their composition $(f\circ g)=h\colon X\rightarrow Y$ is a homeomorphism.