Given that $u=(1,0)$, $v=(0,1)$, $w=(0,-1)$, find the hyperbolic area of the circle inscribed in the triply asymptotic triangle $[u,v,w]$.
My approach (I'm using the Poincaré disc model):
Set up a system of three equations based on the equal hyperbolic distances between the point (say, $a$) of the incentre of the hyperbolic triangle:
$$r=d_H(a,v-w)$$ $$r=d_H(a,w-u)$$ $$r=d_H(a, v-u)$$
(Is this correct?)
Upon solving the system, get $a=\left(\frac23, 0\right)$, so that $$r=\cosh^{-1}\left(\frac{3\sqrt{10}}{5}+1\right)$$
Do you think this is correct?
If $u-w$ means the geodesic from $u$ to $w$ then yes, $a$ is the point that is equidistant from those three geodesics. But I don't know how you set up and solve that system of equations.
My approach: any three points on a circle can be moved to other three points by a Möbius transformation. So, replace the vertices by $(-1/2, \pm\sqrt{3}/2)$ and $(1,0)$, that is the vertices of an equilateral triangle. Then the center of inscribed circle is $(0,0)$, by symmetry. To find its radius, consider that the geodesic between $(-1/2, \pm\sqrt{3}/2)$ is an arc of a circle with its center on the negative real axis. As this circle is perpendicular to the unit circle, we get a right triangle with $60$ and $30$ degrees, where the shorter leg is the radius of the unit circle. So the hypotenuse is $2$, the center is $(-2,0)$, and the other leg is $\sqrt{3}$. It follows that the Euclidean distance from this geodesic to $(0,0)$ is $2-\sqrt{3}$. Hence, the hyperbolic distance is $$ r = 2\tanh^{-1}(2-\sqrt{3})$$ (Depending on your normalization of the metric.) This is the hyperbolic radius of the inscribed disk. Then there's a formula for its area.