I'm trying to find the two points that form the shortest line between two ellipses (can have different semi-major and semi-minor axes, but they are not rotated), such as the green line in the image below. I assume this line would be normal to both ellipses. The red line just connects their centers. The dashed gray line connects where the red line intersects the ellipses. This dashed gray line is longer than the green line.
I understand that finding the start and end points of the green line is mathematically complex and requires numerical techniques. Is there a simple quick method to find the start and end points of the green line if it doesn't have to be exactly the shortest? It must be better than the dashed gray line though. I don't really have a good heuristic or constraint for how inexact it can be.
My initial thought is to start with the intersection points of the red line (connect the centers of the ellipses). Then, for each ellipse, move the point toward the direction that the red line veers from the normal. The greater the angle between the red line and the normal, move the point further.
Let the two ellipses be
$$ \left \{ \begin{array}{rcl} \displaystyle \frac{x^2}{a^2}+\frac{y^2}{b^2} &=& 1 \\ \displaystyle \frac{(x-h)^2}{a^2}+\frac{(y-k)^2}{b^2} &=& 1 \end{array} \right.$$
where $\dfrac{h^2}{a^2}+\dfrac{k^2}{b^2}>4$ for no intersection.
At the minimal distance, the line segment should be normal to both ellipses.
Now considering tangents of the two ellipses with common slope $m$,
$$ \left \{ \begin{array}{rcl} y &= &mx \pm \sqrt{a^2 m^2+b^2} \\ y-k &= &m(x-h) \mp \sqrt{a^2 m^2+b^2} \end{array} \right.$$
The perpendicular distance between two tangents is
$$d=\frac{mh-k\pm 2\sqrt{a^2m^2+b^2}}{\sqrt{1+m^2}}$$
Using the upper case, the local extremum of $d$ as $m$ varying gives the minimal distance.