I tried to draw this to make it hopefully more straightforward to understand.
So here is an angle drawn that demonstrates a small rounded corner:

The bounds of the rounded corner can be drawn like this:

Specifically, what's happening is this. There are two lines that join together at the rounded corner. The corner is an arc, or a "part of a circle" (not sure the correct terminology). It's not an ellipse or an arbitrary shape. The corner is just a segment of a circle connecting to lines that are coming together into the "rounded corner".
The pink circle in this next image shows how you can complete the drawing of the rounded corner into a complete circle:

The only other piece of information I can gather from this diagram is that there's an angle, like here:

So the question is: how to determine the dimensions of this arc (given that we define an arc as a segment of a circle), given two lines that are joined at a specific angle. Specifically, the question is looking for how to solve for:
- radius of the circle
- segment edge length (or some chunk amount of the crust of the circle)
- orientation of the arc
There can be different orientations, as in this image:

Many different radii:

And there can be many different "segment edge lengths"

Sorry for all the images, I would've tried to draw it in tikz but it's not supported.
But the question is, how to calculate those 3 properties given 2 lines intersecting at an angle. Basically, I would like to know how to smoothly join two lines that are coming together at an angle so there's no crease, but also make it so the join is a semi circle of some sort, not an arbitrary curve.
Given points $A$ and $B$ on the rays and point $C$ their intersection: find unit vectors $a=\frac{A-C}{|A-C|}$ and $b=\frac{B-C}{|B-C|}$. Add these two together $c=a+b$ and we find a vector that bisects the opening of the intersection, though it's not a terribly useful size yet.
What we aim to do is find a $c'$ such that it stands off from both $a$ and $b$ by exactly $1$, so we must find out how much it does so now, and divide $c$ by that. For this we can use the law of cosines: we have a triangle with vectors $a,b,c$ and we wish to get angle information from it.
$$\begin{align} c^2&=a^2+b^2-2ab \cos \gamma\\ |c|^2&=1^2+1^2-2(1)(1)\cos\gamma\\ \cos\gamma&=\frac{|c|^2-2}{2} \end{align}$$
$$\begin{align} h&=\sin\gamma\\ &=\sqrt{1-\cos^2\gamma}\\ &=\sqrt{1-\left(\frac{|c|^2-2}{2}\right)^2}\\ \end {align}$$
Now, $c'=c/h$; we also may need to scale $a$ and $b$, and the scaling factor there is $s=\sqrt{|c'|^2-1}$, being the leg of a right triangle with $c'$ as the hypotenuse.
Now all we have to do is scale this by our chosen radius and find these points in the space of our original points, which is easy: the center is at $C+c'r$, and the circle meets the rays at $C+asr$ and $C+bsr$.