It is possible for some non-linear autoencoder to compress the input data into the same dimension as PCA but can preserve more information than PCA does?
2026-03-31 19:08:53.1774984133
Non-linear autoencoder versus linear autoencoder (PCA)
383 Views Asked by Bumbble Comm https://math.techqa.club/user/bumbble-comm/detail At
1
Yes, imagine that your data points lie on a circle. PCA projection down to 1 dimension will position points on opposite sides of the circle next to each other, whereas non-linear projection down to 1 dimension can avoid this problem to some extent (though local neighborhood structure cannot be preserved at the end-points).
Auto-encoders are a broad class of models, so some auto-encoders can do the right thing here but not all. For instance, here's an autoencoder architecture that would not do better than PCA: https://arxiv.org/pdf/1804.10253.pdf