I am aware of manifold learning as a form of non linear dimensionality reduction.
But, can it also be that there exists a higher dimensional manifold (dimensions greater than observed data), which can provide better results for data structure representation/classification/regression compared to original data or a low dimensional manifold?
I am looking for intuitive and theoretical justification of existence of such manifolds, as online search seems to always return results associated with dimensionality reduction, not expansion.
My question is motivated from deep learning perspective, where we can use higher number of nodes in hidden layers compared to input layer, which is akin to expanding the dimensionality of original dataset.