I came across this question that asked the same, but I couldn't understand the reasoning. I feel it wasn't answered clearly, and the OP also sought clarification.
The answer stated,
From a physics perspective -- (here we go again, answering physics questions on a math site! ;) ) -- it may make more sense if you go into the inertial reference frame where the ship being approached is stationary. This involves subtracting the (vector) velocity of the ship being approached from the velocities of both ships. [...] one ship is stationary, its velocity is zero, so it stays at the same place all the time. It's at a point. (The red point.) The other ship moves along some straight line path past that point. The minimum distance from the point to the line is a perpendicular.
But this doesn't explain why $V_{ba} \perp V_{b}$. In fact, I think OP tried to ask the same.
But the question I'm asking John is marked in red above. The two lines in your diagram are perpendicular, the solid blue line represents the velocity vector of the patrol boat B as you rightly said. But this must mean that the dashed blue line is a relative velocity, (as well as being the shortest distance). Can you explain if the dashed line represents the velocity of B relative to S?
This is what I understand so far: (1-3 are covered in the linked question's accepted answer)
- Assume $V_{a}$ is velocity of first body and $V_{b}$ the velocity of the second
- $V_{ba} = V_{b} - V_{a}$ can be taken as object A is stationary and object B moves with $V_{ba}$
- For distance of closest approach, it's as simple as dropping a perpendicular from A to $V_{ba}$
- This part, I don't get (and unexplained in the linked question). For closest approach to happen, $V_{ba} \cdot V_{b} = 0$.
Why must $V_{ba} \perp V_{b}$?
Given $V_{ba}$ and $V_{b}$ are in different reference frames, how do I derive the relation that their dot product must be zero?


We have two boats $A$ and $B$ in the plane with $$A(t)=a+t u,\qquad B(t)=b+t v\qquad(-\infty<t<\infty)\ ,$$ whereby it is assumed that the velocities $u$ and $v$ are linearly independent. We are interested in the vector $$d(t):=B(t)-A(t)=(b-a)+t(v-u)\ .$$ Since $\bigl|d(t)\bigr|\to\infty$ when $|t|\to\infty$ there is a certain $t_*$ for which $|d(t)|$ is minimal, and this $t_*$ can be found by putting $${d\over dt}|d(t)|^2={d\over dt}\bigl(d(t)\cdot d(t)\bigr)=2 d(t)\cdot d'(t)$$ to zero. As $d'(t)=v-u$ it follows that $$d(t_*)\cdot (v-u)=0\ .$$ This means that at the time $t_*$ of "nearest contact" the relative speed of the two boats is orthogonal to the vector $d(t_*)$ connecting the two boats.