Mahalanobis distance connection with Euclidean distance

Question

My question is more about the meaning of Mahalanobis distance when compared to L2 distance. As a reminder:

• Mahalanobis distance is the distance between a point $x$ and a distribution with mean $\mu$ and covariance $\Sigma$ can be calculated as: $d_{\text{Mahalanobis }} = \sqrt{(x-\mu)^T\Sigma^{-1} (x-\mu)}$

Back to the question: if the Euclidean distance between a point $p_a$ and $p_b$ is for example 0.5 and the Mahalanobis distance between the point $p_a$ and some other distribution $\beta$ is 0.75, could we state that point $p_a$ is closer to $p_b$ than distribution $\beta$?

If not, then under what conditions we could have this comparison valid?

Answer 1

Comparing a Euclidean distance directly to a Mahalanobis distance does not make sense, generally.

However, define distribution $\alpha$, such that $\vec{\mu_{\alpha}} = \vec{p_b}$ and $\Sigma_\alpha = I$. So $\alpha$ is centered at $\vec{p_b}$ and spherical.

The Mahalanobis distance between $\alpha$ and $\vec{p_a}$ is

$$ d_M{(\alpha,p_a)} = \sqrt{(\vec{p_a}-\vec{\mu_\alpha})^T\Sigma_\alpha^{-1}(\vec{p_a}-\vec{\mu_\alpha})} = \sqrt{\sum_{i=1}^n(p_{a_i}-p_{b_i})^2}$$

Thus the Euclidean distance between $\vec{p_a}$ and $\vec{p_b}$ is the Mahalanobis distance between $\vec{p_a}$ and $\alpha$.

Now you can compare Mahalanobis distances and make a statement that $\vec{p_a}$ is closer to $\alpha$ (which has center $\vec{p_b})$ than to $\beta$.

Answer 2

I think the Mahalanobis metric is perhaps best understood as a weighted Euclidean metric. For example, consider distances in the plane. Then, the Euclidean metric coincides with one's geometric intuition of distance, and the Mahalanobis metric coincides with costliness of traveling along that distance, say, treating distance along one axis as "more expensive" than distance along the other axis.

So, if the distance between two points if 0.5 according to the Euclidean metric but the distance between them is 0.75 according to the Mahalanobis metric, then one interpretation is perhaps that travelling between those two points is more costly than indicated by (Euclidean) distance alone.

Does this answer?

---

P.S. This interpretation is meaningful in statistics where the general multivariate normal distribution (MVN) can be understood as being derived from the MVN with unit variance by means of a scale transformation: effectively replacing the Euclidean metric in the exponential function with a Mahalanobis metric turns a standard Gaussian distribution into a general one (there is also the mean to consider, but that is secondary).

P.P.S. I'm sure that, in machine learning, there are cases in which using a Mahalanobis metric instead of Euclidean corresponds to more/less heavily penalizing incorrect guesses of one feature than of another. As the other answerer pointed out, Mahalanobis metric absorbs Euclidean as a special case.