Seeking a geometric characterisation of a certain property of a circle, a tangent ellipse within it, and a point strictly within the ellipse.

Question

Consider a circle $C$, an ellipse $E$ strictly within $C$ except for a tangent point $p$, and a point $b$ strictly within $E$. Consider a point $q$ on $E$, and a point $q'$ given by the intersection of the extension of $qb$ with $E$, such that both $q$ and $q'$ are distinct from $p$. Define the points $r$ and $r'$ as the intersection of $C$ with the extensions of $pq$ and $pq'$ respectively.

We are interested in whether the point $j$, the intersection of the extension of $pb$ with $rr'$, lies inside the ellipse.

We can show (algebraically using Mathematica page 1,page 2) that the point $j$ is a property only of $C$, $E$, and $b$, independent of $q$, as illustrated in the figure:

$j$ on $q$" />

So we would like some way to characterize this property (whether $j$ is interior to $E$) of $C$, $E$, and $b$, geometrically without introducing $q$.

Answer 1

If you are interested in the locus of point $b$ when $j$ is inside ellipse $E$, you could reverse your construction: take any point $j$ on $E$, trace from $j$ any line intersecting circle $C$ at $r$, $r'$ (that line is taken parallel to $x$ axis in figure below), join $pr$, $pr'$ to find their intersections $q$, $q'$ with $E$. Point $b$ lies on segment $qq'$. Repeating the construction for another line $tt'$ through $j$ you can find a second segment, whose intersection with $qq'$ is then point $b$ (otherwise, you could also find $b$ as the intersection of $pj$ and $qq'$).

I didn't embark into a computation, but with GeoGebra I could find the locus of $b$ as $j$ varies on the ellipse: it should be another ellipse (pink in figure below), tangent at $P$ with $C$ and having its axes parallel to those of $E$.

Answer 2

Consider an ellipse in standard position with equation $$\frac{x^2}{a^2}+\frac{y^2}{b^2}=1 \tag1$$ and define $c^2 := a^2 - b^2$. At point $T=(a\cos t,b\sin t)$, with $0\leq t\leq \pi$ (to avoid sign complications), the ellipse has unit inward normal $\vec{u}$ and unit clockwise tangent vector $\vec{v}$ given by $$ \vec{u} := \frac1s\left(-b\cos t, -a \sin t\right) \quad \vec{v} := \frac1s\left(a\sin t, -b\cos t\right) \quad s := \sqrt{a^2\sin^2t+b^2\cos^2t} \tag2$$ We can parameterize points on the ellipse thusly: $$E(\theta) = T + \frac{2 a b s^3 \cos\theta }{\left( \begin{array}{c} (c^2 (a^2 \sin^2\theta - b^2 \cos^2\theta)+a^2b^2) \cos^2\theta \\ + a^2 b^2 \sin^2\theta + a b c^2 \sin2t \cos\theta \sin\theta \end{array}\right)}\left( \vec{u} \cos\theta + \vec{v} \sin\theta \right) \tag3$$ where $\theta$ is the angle that $\overrightarrow{TE}$ makes with normal vector $\vec{u}$. (We've effectively converted to a $uv$-coordinate system centered at $T$.) Note that $|TE|$ is the big multiplied factor, which amounts to a "polar" representation of the ellipse in the new coordinate system; the rectangular representation in that system (with $u$ and $v$ giving distances in the $\vec{u}$ and $\vec{v}$ directions from $T$) can be read immediately from the coefficients of $\cos^2\theta$, $\sin^2\theta$, $\sin\theta\cos\theta$ in the denominator, and $\cos\theta$ (with a sign change) in the numerator:

$$\begin{align} u^2 (c^2 (a^2 \sin^2t- b^2 \cos^2t)+a^2b^2) + v^2 a^2 b^2 + u v a b c^2 \sin2t - 2 u a b s^3 = 0 \tag4 \end{align}$$

From this we find that (ignoring special cases) the axes of the ellipse are rotated relative to the $uv$-frame by an angle $\phi$ satisfying $$\tan2\phi = \frac{\text{coeff of}\; uv}{(\text{coeff of}\; u^2) - (\text{coeff of}\; v^2)} = \frac{a b\sin2t}{a^2\sin^2t-b^2\cos^2t} \tag5$$

---

Now, we'll get at OP's construction by considering points $P$ and $Q$ on the ellipse for angles $\theta=p$ and $\theta=q$, and taking $R$ to be the intersection of $\overline{PQ}$ with the line through $T$ making angle $r$ with vector $u$. (This $R$ plays the role of OP's point $b$, with $P$ and $Q$ corresponding to $q$ and $q'$.) We find that $$|TR| = \frac{2 a b s^3}{ \left(\begin{array}{c} (a^4\sin^2t + b^4 \cos^2t) \cos r + a b c^2 \sin2t \sin r \\ + a^2 b^2 ((\tan p + \tan q) \sin r - \tan p \tan q \cos r) \end{array}\right) } \tag6$$ We can manipulate this into $$\begin{align} \frac1{a^2b^2}&\left( \frac{2 a b s^3}{|TR|} - (a^4 \sin^2t + b^4 \cos^2t) \cos r - a b c^2 \sin2t \sin r \right) \\[4pt] &= (\tan p + \tan q ) \sin r - \tan p \tan q \cos r \end{align} \tag7$$

The significance of $(7)$ is that its right-hand side is devoid of ellipse-specific parameters, while its left-hand side is devoid of $P$- and $Q$-specific parameters.

To see why this is helpful, consider another ellipse (generalizing OP's circle) with parameters, say, $\hat{a}$, $\hat{b}$ (and $\hat{c}$), and a point $\hat{T}:=(\hat{a}\cos\hat{t}, \hat{b}\sin\hat{t})$ on it. We can parameterize that ellipse relative to unit normal and tangent vectors $\hat{u}$ and $\hat{v}$ (with $\hat{s}$) at $\hat{T}$. Let's move this new ellipse to be tangent to the first, aligning $\hat{T}$, $\hat{u}$, and $\hat{v}$ with with $T$, $\vec{u}$, and $\vec{v}$. Then we can use the lines through $T=\hat{T}$ with the earlier direction angles $p$, $q$, $r$ (no hats!) determine points $\hat{P}$ (collinear with $T$ and $P$), $\hat{Q}$ (likewise), $\hat{R}$ (likewise) on this new ellipse. The last of these points satisfies a suitably-hatted counterpart to $(6)$, and thus also a counterpart to $(7)$; since the right-hand sides of $(7)$ and this new relation match, their left-hand sides do, too. That is, we have $$\begin{align} \frac1{a^2b^2}&\left( \frac{2 a b s^3}{|TR|} - (a^4 \sin^2t + b^4 \cos^2t) \cos r - a b c^2 \sin2t \sin r \right) \\[4pt] = \frac1{\hat{a}^2\hat{b}^2}&\left( \frac{2 \hat{a} \hat{b} \hat{s}^3}{|T\hat{R}|} - (\hat{a}^4 \sin^2\hat{t} + \hat{b}^4 \cos^2\hat{t}) \cos r - \hat{a} \hat{b} \hat{c}^2 \sin2\hat{t} \sin r \right) \end{align} \tag8$$ In this way, we have confirmed OP's assertion (generalized to not-necessarily-circular ellipses) that the relative positions of $R$ and $\hat{R}$ along the $r$-line are independent of the points along the $p$- and $q$-lines.

For the question-proper, we seek the locus of $R$ when $\hat{R}$ traces the $ab$-ellipse.

That is, when $\hat{R}=E(r)$, so that $|T\hat{R}|$ is the big factor in $(3)$, with $\theta=r$. Substituting this into $(6)$ and solving for $|TR|$ gives $$\begin{align} |TR| &= \frac{-D \cos r}{A \cos^2r + B \cos r\sin r + C \sin^2r} \\[8pt] A &= \hat{a}\hat{b} c^2 (a b \hat{s}^3 + \hat{a} \hat{b} s^3 ) (a^2 \sin^2t - b^2 \cos^2t) \\ &\phantom{=}- a^2 b^2 \hat{c}^2 s^3 (\hat{a}^2 \sin^2\hat{t} - \hat{b}^2 \cos^2\hat{t}) \\ &\phantom{=}+ a^3 b^3 \hat{a} \hat{b} \hat{s}^3 \\[4pt] B &= a b \hat{a}\hat{b} (c^2 ( a b \hat{s}^3 + \hat{a} \hat{b} s^3) \sin2t - a b \hat{c}^2 s^3 \sin2\hat{t}) \\[4pt] C &= a^3b^3 \hat{a}\hat{b}\hat{s}^3 \\[4pt] D &= -2 a b s^6 \hat{a}^2 \hat{b}^2 \end{align} \tag9$$

This is the "polar" representation (in our $uv$ coordinate frame) of yet another ellipse tangent at $T=T'$, as suggested by Intelligenti pauca's sketch. Its rectangular representation (with $u$ and $v$ representing the distances from $T$ in the $\vec{u}$ and $\vec{v}$ directions) is $$A u^2 + B u v + C v^2 + D u + E v + F = 0 \quad (E=F=0)$$

---

Gleaning the exact geometry of this ellipse a bit of a chore in general. Let us simplify to OP's specific case, where the $\hat{a}\hat{b}$-ellipse is a circle; we have $\hat{a}=\hat{b}=\hat{s}$ and $\hat{c}=0$, so that

$$\begin{align} A &= c^2 (a b \hat{a} + s^3 ) (a^2 \sin^2t - b^2 \cos^2t) + a^3 b^3 \hat{a} \\[4pt] B &= a b c^2 ( a b \hat{a} + s^3) \sin2t \\[4pt] C &= a^3b^3 \hat{a} \\[4pt] D &= -2 a b s^6 \end{align} \tag9$$

Then we readily find that this ellipse is rotated relative to the $uv$-frame by $\phi$ satisfying $$\tan2\phi=\frac{B}{A-C} = \frac{ab\sin2t}{a^2\sin^2t-b^2\cos^2t} \tag{10}$$ which matches $(5)$, confirming Intelligenti pauca's suspicion that the axes of this ellipse are parallel to those of the $ab$-ellipse. The center of this ellipse is $$(u,v) = \frac{D}{B^2-4AC}(2C,-B) = \frac{s^2}{2mn} (2a^2 b^2 \hat{a}, -c^2 p\sin2t) \\[16pt] m := a b \hat{a} - c^2 s \cos^2t \qquad n := a b \hat{a} + c^2 s \sin^2t \qquad p := a b \hat{a} + s^3 \tag{11}$$

Rotating the ellipse by $\phi$ to align its axes with $\vec{u}$ and $\vec{v}$ gives

$$u^2 b^2 m + v^2 a^2 n + 2 u a b^2 s^3 \cos t + 2 v a^2 b s^3 \sin t = 0 \tag{12}$$

That is, $$\left(u + \frac{a s^3 \cos t}{m}\right)^2 \frac{m^2 n}{a^3 b \hat{a} s^6} + \left(v + \frac{b s^3 \sin t}{n}\right)^2 \frac{mn^2}{a b^3 \hat{a} s^6} = 1 \tag{12'}$$ From this, we read that the major and minor radii (say, $a'$ and $b'$) of this third ellipse are $$a' = \sqrt{\frac{a^3 b \hat{a} s^6}{m^2 n}} \qquad b' = \sqrt{\frac{a b^3 \hat{a} s^6}{mn^2}} \quad\to\quad (c')^2 := (a')^2-(b')^2 = \frac{a b c^2 \hat{a} p s^6}{m^2n^2} \tag{13}$$ and its eccentricity, $e'$, is $$e' := \frac{c'}{a'} = \frac{c}{a}\; \sqrt{\frac{p}{n}} = e\;\sqrt{\frac{p}{n}} \tag{14}$$ where $e$ is the eccentricity of the $ab$-ellipse.

Determining other properties of these figures is left to the reader. $\square$