I am reading Christopher Bishop's "Pattern Recognition and Machine Learning" and in the first chapter, where he talks about the curse of dimensionality, he gives the following example:
Consider, for example, an application in manufacturing in which images are captured of identical planar objects on a conveyor belt, in which the goal is to determine their orientation. Each image is a point in a high-dimensional space whose dimensionality is determined by the number of pixels. Because the objects can occur at different positions within the image and in different orientations, there are three degrees of freedom of variability between images, and a set of images will live on a three dimensional manifold embedded within the high-dimensional space. Due to the complex relationships between the object position or orientation and the pixel intensities, this manifold will be highly nonlinear. If the goal is to learn a model that can take an input image and output the orientation of the object irrespective of its position, then there is only one degree of freedom of variability within the manifold that is significant.
I haven't took a course on differential geometry before; but I have some informal ideas on what manifolds are. I thought them as low dimensional continuous (not in a rigorous way) subsets of high dimensional Euclidean Spaces. For example, a curved surface of a blanket can be considered as a manifold in the space of a room.
Well, that explanation does not work when it comes to a more rigorous text like in the above. How should I understand the term "manifold" in the context of the above paragraph? I see that there is a high dimensional space, with the dimension equal to the number of pixels in the given images. But the definition of the three dimensional manifold is not exactly a three dimensional ("three pixel" sized images) subset of the image pixels exactly. It sounds like that a transformation function is applied to each point (image) in this high dimensional space and each high dimensional point is mapped to a three dimensional point in some three dimensional space. (Consisting of x,y coordinates and an orientation angle). But I don't see how this is related with a manifold. So, how should I understand the term "manifold" in such a context, in an informal and intuitive way? Is this something completely different from my sloppy understanding of the term?
While it's true that any manifold can be imbedded in some high dimensional Euclidean space, modern mathematics does not see this as the right way to think about them. A manifold is an object such that if you look at a tiny piece of it, it will behave like a Euclidean space, which you can take to mean that you can parameterize it with coordinates. An example is the surface of a planet. A planet is so large compared to a person that the overall spherical nature is not visible to a person on the surface. Rather, the space around the person seems to be essentially flat (barring terrain irregularities).