I need to calculate angle of rotation of an image from X and Y axis, what I'll have is 4 co-ordinates of the corners of the image (basically a perfect square image is rotated in all possible angle), using this co-ordinates I need to calculate the angle of rotation in X and Y axis.
For example if four corner makes a perfect square then the angle of rotation from all axis is 0, lets say if the top edge is shorter than the bottom edge or vice versa then it is rotated along X axis, or if the right edge is shorter that the left edge or vice versa then it is rotated along Y axis, or the combination of above 2 examples with rotation on X and Y axis.
And what if I got the angles of 4 corners of the square image, lets say if the 4 corners of the image is equal to 90 degree, means the rotation along X and Y axis are 0 degrees which is a perfect square.
In the above image, I need to calculate the angle of the Object based on the Glass screen.
Can someone guide me how to do calculation?
You might as well write down the equations. You have 4 points which define 4 "ray" along which the original vertices lie:
$$\vec{r}_A(t_A)=E'+(A'-E')t_A$$ where $t_A$ is unknown. Same for all four points.
Your unknowns are the three parameters of the plane in which the square lies: the normalized normal and the distance from the coordinate origin can be combined into a single nonnormalized vector $\vec{n}$. Let the plane be:
$$(\vec{r}-E')\cdot \vec{n}=1$$ The normal $\vec{n}$ encodes the orientation you're looking for. The distance (inverse magnitude of $\vec{n}$) you probably won't be able to get accurately, because once the square is far enough so that parallax distortion isn't noticeable, the shape is independent of the distance, the problem becomes degenerate.
Ray-plane intersections become $$t_A=\frac{1}{(A'-E')\cdot \vec{n}}$$ $$A=E'+\frac{A'-E'}{(A'-E')\cdot \vec{n}}$$ and the same for $B$, $C$, $D$.
Now you have to solve the equations that ensure this to be a square. Try $$B-A=C-D$$ $$(A-B)\cdot (C-A)=0$$ $$(A-C)\cdot(B-D)=0$$ I deliberately avoided the absolute values as conditions for a square. Note that not all of these 5 equations are independent. Now the question is well posed, you just find a way to solve this system. I'd try numerically - minimization or something.