A heuristic explanation of the Curse of Dimensionality

89 Views Asked by At

From Principles and Theory for Data Mining and Machine Learning, Clarke et al. (2009):

This phrase [the "Curse of Dimensionality"] was first used by Bellman (1961)... The result is that difficulty outstrips conventional data gathering even for what one would expect were relatively benign dimensions. A heuristic way to look at this is to think of real functions of $x$, of $y$, and of the pair $(x, y)$. Real functions $f$, $g$ of a single variable represent only a vanishingly small fraction of the functions $k$ of $(x, y)$. Indeed, they can be embedded by writing $k(x, y) = f(x) + g(y)$. Estimating an arbitrary function of two variables is more than twice as hard as estimating two arbitrary functions of one variable.

How am I supposed to deduce that last sentence? I particularly don't understand the meaning of the "embedding" with the $+$ sign.

I'm sorry if not enough effort is seemingly put into this, but I'm not sure how to display effort when it's an issue of reading comprehension.