How can I decompose the affine transformation:
$$ \begin{bmatrix}1&\text{shear}_x\\\text{shear}_y&1\end{bmatrix}$$
into rotation and scaling primitives?
$$ \begin{bmatrix}\cos\theta&-\sin\theta\\\sin\theta&\cos\theta\end{bmatrix}$$ \begin{bmatrix}\text{scale}_x&0\\0&\text{scale}_y\end{bmatrix}
You can only do this if $\textrm{shear}_x = -\textrm{shear}_y$, in which case
$$ \begin{align} \theta &= \textrm{atan}(\textrm{shear}_y)\\ \textrm{scale}_y&=\sqrt{\textrm{shear}_y^2 + 1}\\ \textrm{scale}_x&=-\sqrt{\textrm{shear}_y^2 + 1}\\ \end{align} $$
To see this we can identify both matrices:
$$ \begin{bmatrix}1& \text{shear}_x \\ \text{shear}_y&1\end{bmatrix} = \begin{bmatrix}\cos\theta&-\sin\theta\\\sin\theta&\cos\theta\end{bmatrix} \begin{bmatrix}\text{scale}_x&0\\0&\text{scale}_y\end{bmatrix} $$
$$ \begin{bmatrix}1& \text{shear}_x \\ \text{shear}_y&1\end{bmatrix} = \begin{bmatrix}\text{scale}_x~\cos\theta&-\text{scale}_y~\sin\theta\\ \text{scale}_x~\sin\theta&\text{scale}_y~\cos\theta\end{bmatrix} $$
Which yields the following equation system:
$$ \begin{eqnarray} 1 &= \text{scale}_x~\cos\theta\\ 1 &= \text{scale}_y~\cos\theta\\ \text{shear}_x &= -\text{scale}_y~\sin\theta\\ \text{shear}_y &= \text{scale}_x~\sin\theta\\ \end{eqnarray} $$
. This can only work if neither $\text{scale}_x$, $\text{scale}_y$, or $\cos\theta$ are zero. If that is the case, then we can immediately see that the scaling is isotropic ($\text{scale}_x=\text{scale}_y=\textrm{scale}$) and that
$$\textrm{scale} = \frac{1}{\cos\theta}$$
. We can use this to simplify the other two equations
$$ \begin{align} \text{shear}_x &= - \frac{1}{\cos\theta}~\sin\theta = - \tan\theta\\ \text{shear}_y &= \frac{1}{\cos\theta}~\sin\theta = \tan\theta\\ \end{align} $$
. Both can only be true at the same time, if $\text{shear}_x = -\text{shear}_y$, yielding above constraint. From here, we can find values for both $\theta$ and $\textrm{scale}$:
$$ \begin{align} \theta &= \textrm{atan}~\textrm{shear}_y\\ \textrm{scale} &= \frac{1}{\cos{\theta}} = \frac{1}{\cos(\textrm{atan}~\textrm{shear}_y)} = \sqrt{\textrm{shear}_y^2 + 1} \end{align} $$