How can dimension reduction lead to better results?

49 Views Asked by At

Can someone please explain why a model fitted using a linear combination of the parameters can have better results (lower error) than a plain vanilla one with all the parameters? Can I think about this like adding a bunch of interaction variables to the model to get to better $R^2$?

1

There are 1 best solutions below

0
On

The hope is to distill something closer to the actual (maybe not directly observed) degrees of freedom controlling the problem, or a few of the most relevant ones. Relationships based on the reduced dimensions might be more stable or easier to interpret, the coefficients less variable across different data sets, and in those or other senses better related to the phenomena.

Other than that, there is the added speed of computation. Smaller record size, faster computation. In some cases the computation with full data cannot be done but is possible on the reduction.