My task is pattern recognition. I need to classify 2D matrices into an arbitrary number of classes.
The question is: For pattern classification, could k nearest neighbours algorithm ever be less accurate that simple pattern averaging.
To illustrate what I mean by pattern averaging:
Let's assume the patterns are 2D binary matrices. Let's also assume there are 4 patterns in total. $\bigl( \begin{bmatrix} 1 & 0\\ 1 & 0\\ \end{bmatrix} \begin{bmatrix} 1 & 0\\ 0 & 0\\ \end{bmatrix} \begin{bmatrix} 0 & 1\\ 0 & 0\\ \end{bmatrix} \begin{bmatrix} 0 & 0\\ 0 & 1\\ \end{bmatrix} \bigr)$
We can now split them into two categories: has 1's only in the first column and has 1's only in the second column. And the pattern averages for the two classes become:
$\bigl( \begin{bmatrix} 1 & 0\\ 0.5 & 0\\ \end{bmatrix} \begin{bmatrix} 0 & 0.5\\ 0 & 0.5\\ \end{bmatrix} \bigr)$
To determine which class a pattern belongs to, we would take simple Euclidean distance.
I'm wondering if the above method could ever be more accurate that knn.