Let $a$ and $b$ be the position vectors of two points $A$ and $B$, and let $p$ be the position vector of another point $P$. Consider the line through the points $A$ and $B$, and show that the shortest distance between $P$ and this line is given by $$ d = \frac{|(p-a) \times (p - b)|}{|b - a|} $$
let $r = a + \lambda (b-a)$ be the line connecting $A$ to $B$, my idea was to find a line connecting the point $P$ to $r$ i.e. $r - p$, then projecting this onto the normal of the line which I cannot find. How can I proceed?
I noticed if I cross these $r - p$ and the normalised version of $b-a$ I get the answer. Why?
Note that
$(p-a) = (p-r) + (r-a) = (p-r) + \lambda(b-a)$
and
$(p-b) = (p-r) + (r-b) = (p-r) + (\lambda-1)(b-a)$.
Then taking the cross product, we get
$(p-a) \times (p-b) = \lambda[(b-a) \times (p-r)] + (1-\lambda)[(b-a) \times (p-r)] = (b-a) \times (p-r)$,
and taking the norm, we get
$|(p-a) \times (p-b)| = |b-a)| |p-r| sin(\theta)$,
where theta is the angle between $(b-a)$ and $(p-r)$.
Hence,
$|p-r| = \frac{|(p-a) \times (p-b)|}{|b-a|sin(\theta)}. $
This is minimized when $\theta = \pi/2$ and
$|p-r| = \frac{|(p-a) \times (p-b)|}{|b-a|} $ .