Must any shortest line between two surfaces coincide with normals to both surfaces?

Question

In differential geometry I know of a result which says something along the lines of

"The point on a surface which is closest to some given point outside must have a line through 
it which goes to the closest point outside so that line is normal to the surface in the point"

If we have two smooth manifolds, then by extension and symmetry I suppose there must exist a line which is normal to both surfaces at the points where they are closest to each other.

Is this correct or what am I missing?

Answer 1

Given the result you have stated, if $M, N$ are two surfaces and $\gamma:[0, 1]\to\mathbb R^n$ be the minimizing curve connecting $M, N$, with $\gamma(0)\in M$ and $\gamma(1)\in N$. Then in particular $\gamma$ is a minimizing curve from $\gamma(0)$ to $N$, so it is perpendicular to $N$ by the result that you have stated; similarly it is perpendicular to $M$.

In fact, it is true in a much more general case:

Let $X$ be a Riemannian manifold, $M, N$ submanifolds of $X$, if $\gamma$ is the shortest curve between $M$ and $N$, then $\gamma$ is perpendicular to both $M, N$.

I will outline a proof in the case of $\mathbb R^n$, which can be generalized to any Riemannian manifolds.

Same setup as above: Suppose $\gamma$ is not perpendicular to $N$ at $\gamma(1)$, then we may find a tangent vector $\vec v\in T_{\gamma(1)}N$ such that $\langle \vec v, \gamma'(1)\rangle>0$. We may find a variation of $\gamma$, i.e. a smooth family of curves, $\alpha:[0, 1]\times(-\varepsilon,\varepsilon)\to\mathbb R^n$ such that $\alpha(t, 0)=\gamma(t)$, $\alpha(0, s) =\gamma(0)\in M$, $\alpha(1,s)\in N$ and $\partial_s \alpha(1,0)=\vec v$. We may think of this as a family of curves connecting $M, N$: $\gamma_s(t) =\alpha(t, s)$. Consider the lengths $L(s) =\int_0^1 |\partial_t\alpha(t, s)|dt$. $$\begin{align} \frac{d}{ds} L(s) & =\int_0^1\frac {\langle\partial_{st} \alpha, \partial_t\alpha\rangle} {|\partial_t\alpha|}dt\\&=\int_0^1 \frac{\partial_t\langle \partial_s\alpha,\partial_t\alpha\rangle-\langle\partial_s\alpha, \partial_{tt} \alpha\rangle}{|\partial_t\alpha|}dt\end{align} $$

Now here we may assume $\gamma$ is parametrized proportional to arclength $|\partial_t\gamma|=\ell$, and also $\alpha(t, 0)=\gamma(t)$ is a geodesic $\partial_{tt}\alpha(t, s)|_{s=0}=0$. So evaluating $\frac{d}{ds} L(s)|_{s=0}$, we get $$L'(0)=\frac 1\ell\langle\partial_t\alpha, \partial_s\alpha\rangle |_0^1=\frac 1\ell \langle \gamma'(0), \vec v\rangle <0$$ So near $s=0$, when $s$ increases, the length decreases, which contradicts with the fact that $\gamma$ minimizes the length between $M, N$. So $\gamma$ must be perpendicular to $N$.

The proof in the case of Riemannian manifold is almost exactly the same. With the partial derivatives replaced by connection, and inner product is given by the Riemannian metric.

Answer 2

Here is a sketch of a proof. Assume you have $S_1$ and $S_2$ two disjoint compact surfaces in $\Bbb{R}^3$. Take $a$ and $b$ in $S_1$ and $S_2$ respectively such that

$$\| b-a\|=\inf_{a^\prime\in S_1,b^\prime\in S_2}\|b^\prime-a^\prime\|.$$

Then $a$ in a critical point of

$$\begin{array}{rcl} S_1&\longrightarrow &\Bbb{R}\\ x&\longmapsto &\|b-x\|^2, \end{array}$$ which is the composition of $S_1\hookrightarrow\Bbb{R}^3$ and $x\in\Bbb{R}^3\mapsto \|b-x\|^2$, which has gradient $2(x-b)$ at $x$. Hence we have $\forall v\in T_aS_1,\ \langle v,2(a-b)\rangle=0$, so $$T_aS_1=Vect(b-a)^\perp$$ which is what you wanted.