I've been reading "The Master Algorithm" by Pedro Domingos, where he uses the following example to demonstrate the difficulty we have with understanding data in higher dimensions than we're used to:
Consider an orange: a tasty ball of pulp surrounded by a thin shell of skin. Let’s say 90 percent of the radius of an orange is occupied by pulp, and the remaining 10 percent by skin. That means 73 percent of the volume of the orange is pulp (0.9^3). Now consider a hyperorange: still with 90 percent of the radius occupied by pulp, but in a hundred dimensions, say. The pulp has shrunk to only about three thousandths of a percent of the hyperorange’s volume(0.9^100). The hyperorange is all skin, and you’ll never be done peeling it!
I'm trying to understand why the volume of the skin isn't 0.1^100. Also, I'm not a math-talking-guy so if you could dumb it all the down for me that would be just marvellous. Thanks.
There is some constant, which we could compute but which we do not actually care about, such that the volume of a hundred dimensional ball of radius $r$ is $Cr^{100}$. Let's say the total radius of your orange is $1$, so the total volume is $C$.
What is the volume of the pulp? Well, by assumption the radius of the pulp is $.9$ so its volume is $C\times .9^{100}\approx C \times 0.000027$. The skin takes up what ever is left! So the volume of the skin is approximately $$C- .000027C = 0.999973C$$ Thus the skin takes up about $99.9973\%$ of the total.