classification modell complexity and distance metrics

35 Views Asked by At

the titel says basically all: Are there differences in modell complexity (Bias-Variance) if someone uses (f.e.) kNN with different metrics? Is there a mathematically background for this? In the web i haven't found anything in this direction. Is the complexity just the same? I'm asking me that because there are really many metrics one could use and I'm interested in a (programmable) logic.

kind regards

1

There are 1 best solutions below

1
On

Yes the choice of distance metric can have an impact on the performance of a KNN model. Different metrics measure closeness in different ways. Consider the L1 and L2 norms. The L1-norm can be said to be less sensitive to outliers and more sensitive to small scale behavior then the l2-norm, focusing on small scale behavior because it does not flatten out around zero like a parabola. It will also be less sensitive to large distances because the slope does not increase with distance from the origin.

This paper might be of interest to you;

Effects of Distance Measure Choice on KNN Classifier Performance - A Review by Prasath et al.