Let $S$ be a circle with centre $O$. A chord $AB$, not a diameter, divides $S$ into two regions $R_1$ and $R_2$ such that $O$ belongs to $R_2$. Let $S_1$ be a circle with centre in $R_1$, touching $AB$ at $X$ and $S$ internally. Let $S_2$ be a circle with centre in $R_2$, touching $AB$ at $Y$, the circle $S$ internally and passing through the centre of $S$. The point $X$ lies on the diameter passing through the centre of $S_2$ and $\angle YXO=30^\circ$. If the radius of $S_2$ is 100 then what is the radius of $S_1$?
I have tried this for over an hour now but I can't get the right answer, which is 60.
After some construction and taking the sine of given angle I got $XY=100\sqrt3$ but radius of circle is still out of reach.

Since $S_2$ touches $S$ internally and passes through the center of $S$, its radius has to be half that of $S$. So since $S_2$ has radius $100$, $S$ has radius $200$.
If a circle touches a line, the radius at the point of contact has to be perpendicular to the line. This tells you that $X$, $Y$ and the center of $S_2$ form a $(30°,60°,90°)$ triangle, with the $60°$ at the center of $S_2$. Therefore that center forms an equilateral triangle with $O$ and $Y$, with an edge length of $100$.
Now assume some coordinates. I'll put $O=(0,0)$ and assume that $AB$ is horizontal, as in the figure in the question. The altitude of an equilateral triangle is $\frac{\sqrt3}2$ its edge length. So in this case $Y=(50\sqrt3,-50)$, with the altitude as $x$ coordinate and half the edge length as $y$ coordinate downwards. Since $XYO$ is a $(30°,30°,120°)$ isosceles triangle, by symmetry you have $X=(-50\sqrt3,-50)$. (This confirms the $\lvert XY\rvert=100\sqrt3$ you found for yourself.)
If you write $C=(-50\sqrt3,-50-r)$ for the center of $S_1$, a circle of radius $r$ around that center will touch $AB$. So now you have to make it touch $S$. To achieve that, you want $200=\lvert OC\rvert+r$ or
\begin{align*} 200-r &= \sqrt{(50\sqrt3)^2+(50+r)^2} \\ (200-r)^2 &= (50\sqrt3)^2+(50+r)^2 \\ 40{,}000 - 400r + r^2 &= 7{,}500 + 2{,}500 + 100r + r^2 \\ 30{,}000 &= 500r \\ r &= 60 \end{align*}