I have a square and a polygon. I want to transform all the points inside this square such that they are mapped inside the polygon. I was trying using scale and rotate matrices but I am not able to come up with anything that is making sense to me. I also googled a lot for some algorithm which can be used to implement this but dint find anything. Can anybody please help me with an algorithm which is there to map the points in square to a polygon?
I want to map the points inside the square to points inside the polygon on the right.
Label the points of your polygon $A$, $B$, $C$, $D$, $E$, starting at the lower left and going counterclockwise. Then let $F$ be the mid-point of the bottom side $AB$. Define a mapping $f:[0,1]\times[0,1]\rightarrow R^2$ by:
$$f(u,v) = (1-u)(1-v)A + (1-u)vF + u(1-v)E + uvD$$
You can check that this maps $[0,1]\times[0,1]$ onto the left half of your polygon (the half with vertices $A$, $F$, $D$, $E$. By suitable scaling, you can get a mapping from the left half of your rectangle to the left half of the polygon.
Do the same thing with the right halves.
Put the two mapping together to get a mapping from the whole rectangle to the whole polygon.
More generally, choose two polylines $P(u)$ and $Q(u)$. These are strings of edges that will serve as the "top" and "bottom" borders respectively. Then use the mapping
$$g(u,v) = (1-v)P(u) + vQ(u)$$
to construct a "ruled" surface between $P$ and $Q$. This maps from the unit square to the polygon. Adjust accordingly to get a mapping from your rectangle to the polygon.