I apologize in advance for perhaps an imprecise formulation of the question.
If I have a point in 1D, it has precisely 2 nearest neighbors independent of choices.
In 2D, if I allow arbitrary orientation then there is an infinite amount of neighbors, but if I discretize on a grid I have either 4 (von Neumann neighborhood) or 8 (Moore neighborhood) or some other choice.
In 3D, arbitrary orientation has a larger infinity (but is it infinitely larger than the 2D case?), and if I discretize on a grid I have 6 (von Neumann neighborhood) or 26 (Moore neighborhood) or some other choice.
In n-dimensions, for example, von Neumann neighborhood goes as 2n, but what are other common choices? What are the typical ways to think about number of neighbors as a function of dimensions in Euclidean space.
If we use a "topology preserving" dimensional reduction (PCA/SOM), then clearly the number of neighbors a point has decreases, what is a good way of thinking of what is preserved?
Thank you!