Given coordinate of 2 points $A,B$ (Cartisian or longitude-latitude coordinate) on sphere of radius $R_1$. Point $O$ is middle of $AB$, 2 points $E$ and $F$ is derivation from $O$ by a distance $d$. The first and second circle are centered at $A$ and $B$ with radius $R$, respectively. The third circle defines by 3 points $C,D,E$. The 4th circles defines by 3 points $C,D,F$. Given a point $X (x,y,z)$, how to test $X$ lies in 4 above circles or not (the white area)? All points lie in sphere and distance between 2 points is the geodesic.
What I need is like an efficient algorithm to check whether or not a point lies in the first or 2nd circle, or intersection of 3th and 4th circle (color white area)?
Input: R_1: radius of sphere,2 points A,B on sphere, radius R and deviation d. A point X(x,y,z) Output: Does X lies in one of 4 circles or not.
My idea is to find center and radius of 2 circles though C,D,E and C,D,F, then given a point X(x,y,z) I can check in order: 1/ Is X in the first circle? 2/ Is X in the 2nd circle? 3/ Is X in the intersection of 3th and 4th circle. I am able to find the great circle distance between 2 point, and its not difficult to find 2 point C,D. But I am unable to figure out how to find coordinate of E and F and the circle on sphere which go through 3 given points.
Does anyone have any new idea? Thank you so much!
As already stated in comments, it might be better to think about this in terms of planes. Suppose $A,B\in\mathbb R^3$ are cartesian coordinates. Then $x=A\times B$ is a vector orthogonal to the plane through $A$, $B$ and the center. $y=A+B$ on the other hand is a vector pointing in the direction of $O$. Now take $\hat x=\frac{x}{\lVert x\rVert}$ and $\hat y=\frac{y}{\lVert y\rVert}$ as the two unit-length vectors in these direcions. You have $O=R_1\hat y$. To get from there to $E$ you have to rotate a bit away from $\hat y$ and towards $\hat x$. How much? Well, the angle to rotation is $\alpha=\frac{d}{R_1}$. So you have $E=R_1(\cos\alpha\,\hat y+\sin\alpha\,\hat x)$ and $F=R_1(\cos\alpha\,\hat y-\sin\alpha\,\hat x)$.
Now you have three points in space, which define a plane. Compute its normal vector using cross products, e.g. $n_3=(C-E)\times(D-E)$. Then the equation of the plane is $n_3\cdot P=n_3\cdot E$, i.e. a point $P$ lies on that plane if its dot product with $n_3$ is the same as for $E$. Likewise $n_4$ with $F$ instead of $E$. Now to describe the interior of these circles, as opposed to the circles themselves, turn them into inequalities in such a way that $O$ satisifies them.