I encountered this problem while trying to determine a generic equation for entasis, but this question is not about entasis.
$\theta$ is wanted—given this lovely figure
given that the two triangles are similar, and given $a$, $b$, and $h$.
I recognize that the sum of the heights of the triangles equals $h$, and that their ratio equals the scale factor, which seems like a likely avenue, but my trigonometry and geometry are weak and I can’t figure this one out.

It is pretty easy to see from the geometry of the figure that
$b \cot \theta + a \cos \theta = h, \tag 1$
whence,
$b \dfrac{\cos \theta}{\sin \theta} + a\cos \theta = h; \tag 2$
now using $\cos^2 \theta + \sin^2 \theta = 1$, i.e. $\cos \theta = \pm \sqrt{1 - \sin^2 \theta}$,
$b \dfrac{\sqrt{1 - \sin^2 \theta}}{\sin \theta} + a \sqrt{1 - \sin^2 \theta} = h; \tag 3$
we choose the positive sign on $\pm \sqrt{1 - \sin^2 \theta}$ since the angle $\theta$ appears to be acute; next, we square:
$b^2 \dfrac{1 - \sin^2 \theta}{\sin^2 \theta} + 2ab \dfrac{1 - \sin^2 \theta}{\sin \theta} + a^2 (1 - \sin^2 \theta) = h^2; \tag 4$
we multiply by $\sin^2 \theta$:
$b^2 (1 - \sin^2 \theta)+ 2ab \sin \theta (1 - \sin^2 \theta) + a^2 \sin^2 \theta (1 - \sin^2 \theta) = h^2 \sin^2 \theta, \tag 5$
which may be written as a quartic equation in $\sin \theta$:
$-a^2 \sin^4 \theta -2ab \sin^3 \theta + (a^2 - b^2 - h^2)\sin^2 \theta + 2ab \sin \theta + b^2 = 0, \tag 6$
or
$a^2 \sin^4 \theta + 2ab \sin^3 \theta + (h^2 - a^2 - b^2)\sin^2 \theta - 2ab \sin \theta - b^2 = 0. \tag 7$
This is about as far as we can push things using elementary algebra and trigonometry. To find $\sin \theta$, we must solve this quartic, which may be done according to this wikipedia page.