I have an XY scanner on a microscope. It requires coordinates of a square's corner to perform a raster scan. During a scan I select a region (child) inside the previous region. Each region can have an arbitrary rotation. My problem is that I can't properly calculate the child's absolute position after rotating and zooming in. I calculate the center of the region for the absolute position as a sum of all previous centers multiplied by the relative size of the square.
For example, I did one scan full range (red), then did another region with rotation (gold), and the selected a new one (green). The relative position on top and bottom doesn't fit to the absolute position. See below:
Offset top

Offset bottom

How can I properly calculate the child's absolute position?
TL;DR: The first set of equations with $LL$, $LR$, $UL$, $UR$ give the coordinates of the corners of a zoomed-in (child) square in the coordinate system of its parent square (gold to red or green to gold in your images). The second such set of equations gives the coordinates of the corners of a twice zoomed-in square in the coordinates of its grandparent (green to red in your images).
Let's relate a child square to its parent with four numbers:
$(x,y)$ - The coordinates of the lower left corner of the child square with respect to the parent square. Both are in the range $[0,1]$.
$\theta$ - The angle of rotation about the child's lower left corner with respect to the parent's lower horizontal side (the local $x$-axis). A positive angle is a rotation in the counter-clockwise direction.
$S$ - The scaling of the square with respect to the parent square. $S=1/2$ means the child square is half the size of the parent. You could also call this the side length of the child square if the parent square has a length of 1. This is in the range $(0, 1)$.
In general, if you have a point $(x_C, y_C)$ in the child square's coordinate system, the following will give the point's coordinates in the parent square's coordinate system (found by drawing pictures and trig): \begin{align}\begin{pmatrix}x_P \\ y_P\end{pmatrix} &= \begin{pmatrix}x + Sx_C\cos\theta - Sy_C\sin\theta \\ y + Sx_C\sin\theta + Sy_C\cos\theta)\end{pmatrix} \\ \\ &= \begin{pmatrix}x \\ y\end{pmatrix} + S\begin{pmatrix}x_C\cos\theta - y_C\sin\theta \\ x_C\sin\theta + y_C\cos\theta\end{pmatrix} \\ \\ &= \begin{pmatrix}x \\ y\end{pmatrix} + S\begin{pmatrix}\cos\theta & -\sin\theta \\ \sin\theta & \cos\theta\end{pmatrix}\begin{pmatrix}x_C \\ y_C\end{pmatrix} \end{align} This is very nearly a simple matrix multiplication, which would make it very easy to compose two such coordinate changes so that a child square could be related to its grandparent. Using affine transformation matrices (link to PDF) and augmenting the coordinates, we can write the transformation this way: $$ \begin{pmatrix}x_P \\ y_P \\ 1\end{pmatrix} = \begin{pmatrix}S\cos\theta & -S\sin\theta & x \\ S\sin\theta & S\cos\theta & y \\ 0 & 0 & 1\end{pmatrix}\begin{pmatrix}x_C \\ y_C \\ 1\end{pmatrix} $$ From these matrices, you can calculate the corners of the child square $C$ in terms of the parent $P$ coordinate system ($LL$ is the lower-left corner, $LR$ is lower-right, $UL$ is upper-left, $UR$ is upper-right): \begin{array}{l} LL = \begin{pmatrix}0 \\ 0\end{pmatrix}_C \rightarrow \begin{pmatrix}x \\ y\end{pmatrix}_P \\ \\ LR = \begin{pmatrix}1 \\ 0\end{pmatrix}_C \rightarrow \begin{pmatrix}x + S\cos\theta \\ y + S\sin\theta\end{pmatrix}_P \\ \\ UL = \begin{pmatrix}0 \\ 1\end{pmatrix}_C \rightarrow \begin{pmatrix}x - S\sin\theta \\ y + S\cos\theta\end{pmatrix}_P \\ \\ UR = \begin{pmatrix}1 \\ 1\end{pmatrix}_C \rightarrow \begin{pmatrix}x + S\cos\theta - S\sin\theta \\ y + S\sin\theta + S\cos\theta\end{pmatrix}_P \end{array} In your images, these equations give the corners of the gold square in the red square's coordinate system and the corners of the green square in the gold square's coordinate system.
Now, we need a composition rule. Consider three squares: $G$, $P$, and $C$ (Grandparent, Parent, and Child). In your images, $G$ is the red square, $P$ is the gold square, and $C$ is the green square. If $P$ has the definition $P_G = [x_1, y_1, \theta_1, S_1]$ with respect $G$, and $C$ has the definition $C_P = [x_2, y_2, \theta_2, S_2]$ with respect to $P$, what is the definition of $C$ in terms of $G$? If a transformation can be represented by a matrix, a composition of transformations (zooming in twice, in our case) is found by multiplying the matrices. \begin{align} \begin{pmatrix}x_G \\ y_G \\ 1\end{pmatrix} &= \begin{pmatrix}S_1\cos\theta_1 & -S_1\sin\theta_1 & x_1 \\ S_1\sin\theta_1 & S_1\cos\theta_1 & y_1 \\ 0 & 0 & 1\end{pmatrix}\begin{pmatrix}x_P \\ y_P \\ 1\end{pmatrix} \\ \\ &= \begin{pmatrix}S_1\cos\theta_1 & -S_1\sin\theta_1 & x_1 \\ S_1\sin\theta_1 & S_1\cos\theta_1 & y_1 \\ 0 & 0 & 1\end{pmatrix}\begin{pmatrix}S_2\cos\theta_2 & -S_2\sin\theta_2 & x_2 \\ S_2\sin\theta_2 & S_2\cos\theta_2 & y_2 \\ 0 & 0 & 1\end{pmatrix}\begin{pmatrix}x_C \\ y_C \\ 1\end{pmatrix} \\ \\ &=\begin{pmatrix}S_1 S_2 \cos(\theta_1 + \theta_2) & -S_1 S_2\sin(\theta_1 + \theta_2) & S_1(x_2\cos\theta_1 - y_2\sin\theta_1) + x_1 \\ S_1 S_2 \sin(\theta_1 + \theta_2) & S_1 S_2\cos(\theta_1 + \theta_2) & S_1(x_2\sin\theta_1 + y_2\cos\theta_1) + y_1 \\ 0 & 0 & 1\end{pmatrix}\begin{pmatrix}x_C \\ y_C \\ 1\end{pmatrix} \end{align}
So, the definition of $C$ with respect to $G$ is given by $$C_G = \begin{bmatrix} x_2' \\ y_2' \\ \theta_2' \\ S_2' \end{bmatrix} = \begin{bmatrix} S_1(x_2\cos\theta_1 - y_2\sin\theta_1) + x_1 \\ S_1(x_2\sin\theta_1 + y_2\cos\theta_1) + y_2 \\ \theta_1 + \theta_2 \\ S_1 S_2 \end{bmatrix} $$
Finally, the corners of the child square in the grandparent's coordinate system are \begin{array}{l} LL = \begin{pmatrix}0 \\ 0\end{pmatrix}_C \rightarrow \begin{pmatrix}S_1(x_2\cos\theta_1 - y_2\sin\theta_1) + x_1 \\ S_1(x_2\sin\theta_1 + y_2\cos\theta_1) + y_2\end{pmatrix}_G \\ \\ LR = \begin{pmatrix}1 \\ 0\end{pmatrix}_C \rightarrow \begin{pmatrix}S_1 S_2 \cos(\theta_1 + \theta_2) + S_1(x_2\cos\theta_1 - y_2\sin\theta_1) + x_1 \\ S_1 S_2 \sin(\theta_1 + \theta_2) + S_1(x_2\sin\theta_1 + y_2\cos\theta_1) + y_1\end{pmatrix}_G \\ \\ UL = \begin{pmatrix}0 \\ 1\end{pmatrix}_C \rightarrow \begin{pmatrix}-S_1 S_2\sin(\theta_1 + \theta_2) + S_1(x_2\cos\theta_1 - y_2\sin\theta_1) + x_1 \\ S_1 S_2\cos(\theta_1 + \theta_2) + S_1(x_2\sin\theta_1 + y_2\cos\theta_1) + y_1\end{pmatrix}_G \\ \\ UR = \begin{pmatrix}1 \\ 1\end{pmatrix}_C \rightarrow \begin{pmatrix}S_1 S_2 \cos(\theta_1 + \theta_2) - S_1 S_2\sin(\theta_1 + \theta_2) + S_1(x_2\cos\theta_1 - y_2\sin\theta_1) + x_1 \\ S_1 S_2 \sin(\theta_1 + \theta_2) + S_1 S_2\cos(\theta_1 + \theta_2) + S_1(x_2\sin\theta_1 + y_2\cos\theta_1) + y_1\end{pmatrix}_G \end{array} In your images, these equations give the coordinates of the green square's corners in the red square's coordinate system.