I am trying to play around applying transformations to rectangle.
In this picture, I skewed a box by -30 degrees on the Y axis to get the following. The purple box is the transformed box and the dashed border, transparent box is the original one.
I was thinking if I can achieve the same by first skewing on the X axis and then rotating the box to compensate for the skewing done.
But I got a very different result.
At first, it's probably because or the order of those transformations done so I flipped them but this time I got another different result. Essentially the opposite of the previous one.
What I am unable to understand is that from Picture 2, it seemed like skewX and rotate by the same amount would work but it didn't.
Can someone please explain why?
A skew by angle $\theta$ in the direction of the $y$-axis (=your first skew) is a linear transformation with the matrix $$ S_y=\left(\begin{array}{cc}1&0\\ \tan\theta&1\end{array}\right). $$ (This is assuming that the origin is a fixed point. If not, then you get an affine transformation instead, but that won't change the following argument in any essential way.)
Similarly, a skew in the direction of the $x$-axis is given by the matrix $$ S_x=\left(\begin{array}{cc}1&\tan\theta\\ 0&1\end{array}\right). $$
For its part, rotation by $\theta$ is represented by the matrix $$ R=\left(\begin{array}{cc}\cos\theta&-\sin\theta\\ \sin\theta&\cos\theta\end{array}\right). $$
The combined transformation of an $x$-skew followed by a rotation is thus given by the matrix product $$ RS_x=\left(\begin{array}{cc}\cos\theta&\sin\theta-\sin\theta\\ \sin\theta&\sin^2/\cos\theta+\cos\theta\end{array}\right)=\left(\begin{array}{cc}\cos\theta&0\\ \sin\theta&1/\cos\theta\end{array}\right), $$ where I used the trig identities $\tan\theta=\sin\theta/\cos\theta$ and $\sin^2\theta+\cos^2\theta=1$ to simplify the matrix.
You see that this matrix product is different from the one representing the $y$-skew, and that's what you are seeing.
Do observe the second column of the matrix of the combined transformation. Its top entry is equal to zero. This means that the combined transformation does map the $y$-axes to itself. But you see from the bottom entry of that second column that the combination stretches the distance between points on the $y$-axis by a factor of $1/\cos\theta.$ This factor is $>1$ when $\theta$ is non-trivial.
The difference between $1$ and $1/\cos\theta$ shows in your images in that the left vertical side of the purple parallelogram in the last image is longer thatn the height of the rectangle.
On the other hand, if you first rotate and the $x$-skew, you get the matrix product $$ S_xR=\left(\begin{array}{cc}1/\cos\theta&0\\ \sin\theta&\cos\theta\end{array}\right). $$ Again, the $y$-axis is mapped onto itself, but this time the stretching factor is $\cos\theta <1$. You see this in your third image.
It may be worth pointing out that the determinants of all these matrices are equal to one. This reflects the fact that the total area of the figure stays the same in all skews and rotations even though individual sides of a rectangle will stretch or shrink as case may be.