I'm trying to figure out which segments of a given conic section curve lie inside a given triangle.
The specific triangle I’m dealing with is the one where $x\ge 0$, $y\ge 0$ and $x+y \le 1$, as shown in yellow in the picture.
I also have a conic section curve (the pale green thing), and I know its equation, in the form $$ ax^2 + by^2 + 2hxy +2fx + 2gy + c =0 $$ The example picture shows an ellipse, but the curve could be a parabola or hyperbola instead, of course.
I know how to calculate the intersection points with the sides of the triangle. There might be zero, two, four, or six of these. The example picture shows six intersections — the red points $A$, $B$, $C$, $D$, $E$, $F$. Now consider all 15 possible pairs of points, like $(A,B)$, $(A,C)$, $(A,D)$ $,\dots,$ $(E,F)$. I want to know which of these pairs correspond to "good" segments of the conic, i.e. ones that lie inside the triangle, as shown in blue in the picture.
In my example, the pairs $(A,B)$, $(C,E)$, and $(D,F)$ are the good ones. $(A,C)$ is bad because it's outside the triangle, and $(A,E)$ is bad because it passes outside the conic at point $C$.
How do I identify the "good" pairs? I think there are at most three of them.
Motivation (if you need some): this is a part of a new “slicing” algorithm for 3D printing.
Added:
People keep assuming the intersection configuration shown in my original picture. My fault, possibly. Here are a few other possible ones as food for thought:
We have to assume a few things first. We assume that you know the coordinates of the three vertices of the right-angled triangle $UVW$. It is also necessary to know that pairs of points ($A$ and $B$), ($C$ and $D$), and ($E$ and $F$) lie on the sides $UV$, $VW$, and $WU$ respectively.
Now, you need to do some calculations. Start by computing the distances $UA$ and $UB$. Our aim here is to find which point, $A$ or $B$, is closer to the vertex $U$. If $UA \lt UB$, then $A$ is closer to $U$ than $B$ and vice versa. Let’s say that $A$ is closer to $U$, then we know that $B$ is closer to the vertex $V$ than $A$. In a similar vein, find out which remaining four points of intersection are closer to which vertices $U$, $V$, and $W$ by computing the distances $VC$, $VD$, $WE$, and $WF$.
Assume that we have found out that the two points of intersection $A$ and $F$ are closer to $U$. Then, that means the arc $AF$ is inside the triangle. Continue in this manner to find the other two arcs, which lie inside the triangle.
To keep calculations simple and easy, you can place the vertex $V$ of the triangle at the origin of the coordinate system if possible.
Now, tell us whether this method is correct, feasible, accurate, and efficient enough.