Construct an affine transformation given the image of 2 points without skewing

Question

I am attempting to create a mobile app in which the user can interact with content on the screen by using two fingers to translate, scale and rotate the content - very much like the interaction with the map in the Google Maps mobile app.

Intuitively, I am convinced that the added constraint that no skewing operations are used to construct the transformation should result in a unique transformation. I compare it to using two fingers to move a piece of a paper on a flat surface - if one can imagine a piece of paper that can be stretched as well, the comparison is complete.

Given the transformation matrix

$$T = \begin{bmatrix} a & b & 0 \\ c & d & 0 \\ t_x & t_y & 1 \end{bmatrix}$$

In the general case, this results in 6 unknowns ($a, b, c, d, t_x, t_y$) and only 4 equations (one for each $x$ and $y$ for each of the two known points). My current attempt is adding the additional constraints that $a = d$ and $b = c$, since the scaling operations I am interested in scale $a$ and $d$ by the same amount, and rotation operations result in $b = c$ or $b = -c$. This gives me 4 equations in 4 unknowns ($a = d, b = c, t_x, t_y$) which I can solve.

Specifically, given an unknown desired transformation $T$, and known points $A$, $A'$, $B$ and $B'$:

$$\begin{align}A = (a_x, a_y) \\ T(A) = A' = (a'_x, a'_y) \\ B = (b_x, b_y) \\ T(B) = B' = (b'_x, b'_y)\end{align}$$

I can evaluate the system of equations

$$\begin{bmatrix}a_x & a_y & 1 & 0 \\ b_x & b_y & 1 & 0 \\ a_y & a_x & 0 & 1 \\ b_y & b_x & 0 & 1 \end{bmatrix} \begin{bmatrix} a = d \\ b = c \\ t_x \\ t_y \end{bmatrix} = \begin{bmatrix} a'_x \\ b'_x \\ a'_y \\ b'_y \end{bmatrix}$$

The result of this is scaling and translations work as expected. However, instead of a rotation, I end up with a symmetrical skewing effect that you can see in the below images. The 6 orange numbers at the top of each image represent the first two columns of the transformation matrix; namely, $\begin{bmatrix} a & c & t_x \\ b & d & t_y \end{bmatrix}$:

Initial identity transformation:

After rotating two fingers clockwise a small amount:

After rotating two fingers clockwise a larger amount

I am sure this is simply a result of my lack of understanding of how to calculate this new transformation matrix correctly.

How can I calculate this new transformation matrix?

Thanks in advance for any help!

Answer 1

In case anyone finds this in the future, I have solved my problem with a different approach. Instead of attempting to calculate the entries in the transformation matrix using a system of equations, I now construct a transformation as the concatenation of four simple transformations.

Given an unknown desired transformation $T$, and known points $A$, $A'$, $B$ and $B'$:

$$\begin{align}A = (a_x, a_y) \\ T(A) = A' = (a'_x, a'_y) \\ B = (b_x, b_y) \\ T(B) = B' = (b'_x, b'_y)\end{align}$$

I construct $T$ as follows:

1. Translating the identity transformation by $-A$
2. Scaling the resulting transformation by the ratio $\overline{B'A'}\over\overline{BA}$
3. Rotating the resulting transformation by $\angle B'A' - \angle BA$
4. Translating the resulting transformation by $A'$

Intuitively, 1 translates the line segment $AB$ moving $A$ to the origin, 2 and 3 scale and rotate this segment to be the same length and angle as the line segment $A'B'$, and 4 translates this segment moving the origin to $A'$. In this way, we know $T(A) = A'$ and $T(B) = B'$. Further, since it is a composition of translations, rotations and scaling, we can be assured that the resulting transformation does not contain any shearing.

With a few small caveats surrounding implementation, this has exactly the effect I am looking for - two fingers can be used to move, scale and rotate the content on the screen.

Answer 2

The skewed results arise from using the wrong constraint for the rotation, namely $b = c$, whereas rotations always give $b = -c$.

Answer 3

Based on my own struggles with the problem, here's what I've come up with:

The Mission:

Given two original points $A,B$ and their projections $A',B'$, compute an affine transformation matrix $T$ such that:

$$A = (a_x,a_y) \\ B=(b_x,b_y) \\ A'=(a_x',a_y') = T(A) \\ B'=(b_x',b_y') = T(B) $$

with the additional constraint that the transformation shall comprise only rotation, scaling and translation, i.e. have the following form:

$$T = \begin{bmatrix} r \cos \theta & r \sin \theta & 0 \\ -r \sin \theta & r \cos \theta & 0 \\ t_x & t_y & 1 \end{bmatrix}$$

(Note that we don't actually care about $\theta$ itself, just its sine and cosine, or possibly even just their respective products with $r$.)

The result:

It is helpful to first compute the vector from one point to the other, both in the original and the projection:

$$ \overrightarrow v = (v_x, v_y) := \overrightarrow{AB} = (b_x - a_x, b_y - a_y) \\ \overrightarrow {v'} = (v_x', v_y') := \overrightarrow{A'B'} = (b_x' - a_x', b_y' - a_y') $$

We also need the length of those vectors:

$$ d := \|\overrightarrow {v}\| = \sqrt{v_x^2+v_y^2} \\ d' := \|\overrightarrow {v'}\| = \sqrt{v_x'^2+v_y'^2} $$

The scaling term $r$ is simply the quotient of the distances between the points after and before transformation, respectively, i.e.:

$$r = \frac{\|\overrightarrow {A'B'}\|}{\|\overrightarrow {AB}\|} = \frac{d'}{d}$$

The rotational terms can be computed based on the fact that $\theta$ is the angle between the original vector from one point to the other, and the corresponding vector in the projection, i.e.:

$$ \theta = \angle \overrightarrow {AB} \overrightarrow {A'B'} = \angle \overrightarrow {v} \overrightarrow {v'} \Rightarrow \\ \cos \theta = \frac{\overrightarrow {v} \cdot \overrightarrow{v'}}{\|\overrightarrow {v}\|\|\overrightarrow {v'}\|} = \frac{v_x v_x' + v_y v_y'}{dd'} $$ $$ \\ \sin \theta = \frac{\overrightarrow {v} \times \overrightarrow{v'} }{\|\overrightarrow {v}\|\|\overrightarrow {v'}\|} = \frac{v_x v_y' - v_y v_x'}{dd'} $$

Typically there is no need to keep track of the scaling and rotation terms separately, and the combined terms can instead be computed as:

$$ r \cos \theta = \frac{v_x v_x' + v_y v_y'}{d^2} \\ r \sin \theta = \frac{v_x v_y' - v_y v_x'}{d^2} $$

(Note that in this case, we never need to compute $d'$.)

Finally, the translation terms of the matrix can be computed simply by transforming one of the points without translation, and comparing the result with the actual projected point; the difference directly gives the translation terms:

$$ t_x = a_x' - (a_x r \cos \theta - a_y r \sin \theta) \\ t_y = a_y' - (a_x r \sin \theta + a_y r \cos \theta) $$

(or correspondingly for $b_x, b_y, b_x', b_y'$.)

I leave it as an exercise to the reader to derive formulae for the translational terms that don't rely on computing the other terms of the matrix first.