How to show that any rectangle in ellipse must be oriented parallel to axes?

Question

A problem which is often given as an exercise for students learning about calculus and finding extrema, is to find maximal possible area of a rectangle inside an ellipse. Such question was asked, for example, here: Find the area of largest rectangle that can be inscribed in an ellipse (A similar problem in three dimensions is also often asked: Dimensions of a box of maximum volume inside an ellipsoid.)

The solution usually starts by stating that we the rectangle must be oriented in such way that the sides are parallel to the ellipse, which gives a simple expression for the area.

Even though the fact that any rectangle inscribed in an ellipse must be oriented in this way seems intuitively clear, I would like to see an argument showing that this is indeed the case. (I have posted as an answer my attempt using analytic geometry.)

Of course, there is one obvious exception. Rectangle inscribed in a circle can be rotated in any direction we want. So we should assume that the semiaxes of the ellipse have different lengths.

Here is a picture illustrating the situation (shamelessly stolen from this post):

Answer 1

Without loss of generality $a>b$. Take an inscribed rectangle with two sides gradient $k\ne0$ and two sides gradient $-\frac{1}{k}\ne0$. Shrink along the $x$-axis by a factor $\frac{b}{a}$. The two sides gradient $k$ now have gradient $\frac{ka}{b}$ and the two sides gradient $-\frac{1}{k}$ now have gradient $-\frac{a}{kb}$. So the rectangle is now a non-rectangular parallelogram with opposite angles not summing to $\pi$. But it is inscribed in a circle. Contradiction.

Answer 2

Suppose we have a rectangle which has all four vertices on the ellipse. If $C$ is the center of the rectangle, then all vertices of the rectangle are in the same distance from $C$. If we choose coordinate system with origin in the point $C$, we get the equation $$x^2+y^2=r^2.$$ We also want to express the condition that the vertices belong to the ellipse. So far we have only chosen the origin of our Cartesian coordinate system to be the point $C$. We can also choose the axes to be parallel to the axes to the ellipse. If we denote by $(x_0,y_0)$ the center of the ellipse, we get the equation $$\frac{(x-x_0)^2}{a^2}+\frac{(y-y_0)^2}{b^2}=1.$$ Since rectangle is centrally symmetric w.r.t. the point $C=(0,0)$, the vertices also lie on an ellipse which is centrally symmetric to the ellipse described above. This si precisely the ellipse $$\frac{(x+x_0)^2}{a^2}+\frac{(y+y_0)^2}{b^2}=1.$$

So we got the following three equations: \begin{align} x^2+y^2&=r^2 \tag{1}\\ b^2(x-x_0)^2+a^2(y-y_0)^2&=a^2b^2 \tag{2}\\ b^2(x+x_0)^2+a^2(y+y_0)^2&=a^2b^2 \tag{3} \end{align} We are asking for the conditions on $x_0$, $y_0$ and $r$ such that there are at least four solutions. Notice that by subtracting the equations $(2)$ and $(3)$ we get $$b^2xx_0+a^2yy_0=0.\tag{4}$$

Let us first show that $x_0=y_0=0$. Assume that this is not true. For example, let us assume $x_0\ne0$. Then we get $$x=\frac{a^2y_0}{b^2x_0}\cdot y. \tag{5}$$ By plugging $(5)$ into $(1)$ we get $$y^2\left(1+\frac{a^2y_0}{b^2x_0}\right)^2=r^2 \tag{6}$$ which gives us two possibilities for $y$. Combining this with $(5)$ we get two possibilities for the point $(x,y)$.

So we see that if $x_0\ne0$, then there are at most two solutions of the above system. The case $y_0\ne0$ is symmetric. In order to get a rectangle we need at four points, so we necessarily have¹ $$x_0=y_0=0.\tag{7}$$ In the other words, we have found out that the center of the rectangle and the center of the ellipse coincide.

This condition significantly simplifies the original system to \begin{align} x^2+y^2&=r^2 \\ b^2x^2+a^2y^2&=a^2b^2 \end{align} From the first equation we get $y^2=r^2-x^2$. By plugging this value into the second equation we get \begin{align} b^2x^2+a^2(r^2-x^2)&=a^2b^2\\ (b^2-a^2)x^2&=a^2(b^2-r^2) \end{align} Since we do not consider the case $b=a$, which corresponds to the ellipse being circle, we get $$x^2=\frac{a^2(b^2-r^2)}{b^2-a^2}.$$ Assuming the RHS is positive, this gives us two possible values $$x=\pm\sqrt{\frac{a^2(b^2-r^2)}{b^2-a^2}}.$$ From these values we can get $$y=\pm\sqrt{r^2-x^2}$$ (again assuming that $r^2-x^2>0$).

Clearly, we got a rectangle with the sides parallel to the axes. (The vertices of rectangle have the form $(x_1,y_1)$, $(-x_1,y_1)$, $(-x_1,y_1)$, $(-x_1,-y_1)$.)

---

In fact, using the above equations we can also find the conditions on $a$, $b$ and $r$ when the intersection exists. W.l.o.g. we can assume $0<a<b$. (The other case is symmetric.) If we want $a^2(b^2-r^2)/(b^2-a^2)$ to be positive, we need $r<b$.

We also need $$0<r^2-x^2 = r^2- \frac{a^2(b^2-r^2)}{b^2-a^2} = \frac{a^2(b^2-r^2)}{b^2-a^2} = \frac{b^2(r^2-a^2)}{b^2-a^2}$$ which gives us $a<r$.

So we get a solution only if $$a<r<b$$ as expected.

---

¹Another possibility how to view the case $(x_0,y_0)\ne(0,0)$, i.e. the case when the center of the rectangle is different from the center of the ellipse, is that in such case we can only get a line segment - which could be considered as a "rectangle having zero width".

Answer 3

EDIT: This picture illustrated the basic idea of the proof below. (Although it is written using the parametrization of ellipse, it roughly corresponds to the picture.) I would say that the idea behind almagest's answer is similar.

The basic idea can be explained like this: Relation between ellipse and circle is that we scaled the coordinate along one of the axes using some factor. This is an affine transformation, so it does not influence whether two vectors are parallel. So this gives us one-to-one correspondence between parallelograms in the circle and parallelograms in the ellipse. Every parallelogram inscribed in a circle is a rectangle. However, if the rectangle does not have sides parallel to the axes, this transformation changes the right angle to a different angle.

Although I only realized this later. (And I have to admit that the way the argument is written out below is much clumsier than the idea illustrated by this picture.)

---

We can use the well-known parametrization of ellipse given by \begin{align*} \newcommand{\vp}{\varphi} x&=a\cos\vp\\ y&=b\sin\vp \end{align*}

This means that any four points on the ellipse can be expressed as $A=(a\cos\vp_1,b\sin\vp_1)$, $B=(a\cos\vp_2,b\sin\vp_2)$, $C=(a\cos\vp_3,b\sin\vp_3)$, $D=(a\cos\vp_4,b\sin\vp_4)$, where $0\le\vp_1<\vp_2<\vp_3<\vp_4$.

If they should form the rectangle, then we necessarily have $\overrightarrow{AB}=\overrightarrow{DC}$ and $\overrightarrow{AD}=\overrightarrow{BC}$. This condition gives us \begin{align*} a(\cos\vp_2-\cos\vp_1)&=a(\cos\vp_3-\cos\vp_4)\\ b(\sin\vp_2-\sin\vp_1)&=b(\cos\vp_3-\sin\vp_4)\\ a(\cos\vp_3-\cos\vp_2)&=a(\cos\vp_4-\cos\vp_1)\\ b(\sin\vp_3-\sin\vp_2)&=b(\cos\vp_4-\sin\vp_1) \end{align*} Of course, this is equivalent to \begin{align*} \cos\vp_2-\cos\vp_1&=\cos\vp_3-\cos\vp_4\\ \sin\vp_2-\sin\vp_1&=\cos\vp_3-\sin\vp_4\\ \cos\vp_3-\cos\vp_2&=\cos\vp_4-\cos\vp_1\\ \sin\vp_3-\sin\vp_2&=\cos\vp_4-\sin\vp_1 \end{align*} This means that the points given by the angles $\vp_1,\dots,\vp_4$ on the unit circle form a parallelogram.¹ Hence we get $\vp_2-\vp_1=\vp_4-\vp_3$ and $\vp_3=\pi+\vp_1$.

If we denote $\vp=\frac{\vp_1+\vp_2}2$ and $\alpha=\frac{\vp_2-\vp_1}2$, then wee can rewrite our four points as \begin{align*} A&=(a\cos(\vp-\alpha),b\sin(\vp-\alpha))\\ B&=(a\cos(\vp+\alpha),b\sin(\vp+\alpha))\\ C&=(-a\cos(\vp-\alpha),-b\sin(\vp-\alpha))\\ D&=(-a\cos(\vp+\alpha),-b\sin(\vp+\alpha))\\ \end{align*} In this way we have describe all parallelorams inscribed in an ellipse.

In order to get a rectangle, the length of the diagonals must be the same. This gives us the condition \begin{align*} a^2\cos^2(\vp-\alpha)^2+b^2\sin^2(\vp-\alpha)&=a^2\cos^2(\vp+\alpha)^2+b^2\sin^2(\vp+\alpha)\\ a^2(\cos\vp\cos\alpha+\sin\vp\sin\alpha)^2+b^2(\sin\vp\cos\alpha-\cos\vp\sin\alpha)^2&= a^2(\cos\vp\cos\alpha-\sin\vp\sin\alpha)^2+b^2(\sin\vp\cos\alpha+\cos\vp\sin\alpha)\\ a^2\cos\vp\cos\alpha+\sin\vp\sin\alpha&=b^2\cos\vp\cos\alpha+\sin\vp\sin\alpha\\ (a^2-b^2)\cos\vp\cos\alpha\sin\vp\sin\alpha&=0\\ \cos\vp\cos\alpha\sin\vp\sin\alpha&=0\\ \sin2\vp\sin2\alpha&=0 \end{align*} (Notice that we have used $a^2-b^2\ne0$.)

So we either get $\sin2\vp=0$, which means $\vp=k\frac\pi2$. (And $\alpha$ is arbitrary.) This solution leads to a rectangle with sides parallel to axis.

The other possibility is $\sin2\alpha=0$, which means $\alpha=k\frac\pi2$. In this case two pairs of points will coincide and we will get line segments instead of rectangles.

---

¹The whole argument can be viewed geometrically. We simply "stretch" the ellipse to a circle. The shear transformation does not change which vectors are parallel to each other. The argument is relying on the fact that we "know" what the parallelograms inscribed in a circle look like. (Maybe it would be interesting to check whether the equalities $\vp_2-\vp_1=\vp_4-\vp_3$ and $\vp_3=\pi+\vp_1$ can also be derived by some algebraic manipulation and trigonometric identities instead of a geometric argument involving unit circle and inscribed parallelogram.)