I'm struggling to see for a method to start this question.
It looks like a question related with mobius transforms.
We have studied about determining the mobius transform when points from the domain are given. For example, getting the cross ratio.
But here I don't see any points from the domain and also a method to determine the center when the mobius transform is given.
Help would be appreciated
With $z = x +iy$ you have the three equations
$\lvert z - 0 \rvert = r$. i.e. $x^2 + y^2 = r^2$
$\lvert z - 1 \rvert = r$, i.e. $(x - 1)^2 + y^2 = x^2 - 2x + 1 + y^2 = r^2$
$\lvert z - (2+i) \rvert = r$, i.e. $(x - 2)^2 + (y - 1)^2 = (x - 2)^2 + y^2 - 2y + 1 = r^2$
Subtracting 2.from 1. you get $2x - 1 = 0$, hence $x = \frac{1}{2}$. Thus $y^2 = r^2 - \frac{1}{4}$. Inserting this in 3. yields $\frac{9}{4} + r^2 - \frac{1}{4} - 2y +1 = r^2$ which means $y = \frac{3}{2}$.
Then you get $\frac{1}{4} + \frac{9}{4} = r^2$, i.e. $r = \sqrt{\frac{5}{2}}$.