This is a question in the Princeton online test of GRE general book. I got it wrong, but even when I looked at the answer I find it difficult to understand how the answer is obtained. In the explanation given, how does if the new angle were $90^\circ$, Quantity $A$ become $\frac{1}{2}$? Wouldn't it be $\frac{\sqrt{2} r}{2r} = \frac{1}{\sqrt2}$?
Then I don't understand how it is concluded that it is less than $60^\circ$
I agree that the explanation is confusing, but the answer is right. If arc $AB$ is shorter than both arc $AC$ and arc $BD$ (and those latter two are equal), then arc $AB$ must be less than $60$ degrees. If we let the diameter $CD = 2$, then the length of the chord $AB$ must be less than $2 \sin (60/2) = 2 \sin 30 = 1$*. Therefore $AB/CD < 1/2$, and the answer (B) is obtained.
*An explanation of this part: Construct the perpendicular bisector of $\overline{AB}$. This is a (vertical) diameter of the circle. Since the arc $AB$ spans less than $60$ degrees, each half of that arc spans less than $30$ degrees. But each half of the line segment $\overline{AB}$ is equal to the sine of half the arc, and is thus less than $\sin 30$. The two halves (which is the entire line segment $\overline{AB}$) is thus less than twice that, or $2 \sin 30 = 1$.