When fitting (let's say) a linear regression model, it is always true, that the more variables we include in our model, the better fit is (in R^2 sense)? I don't want to discuss here overfitting, problems with diagnostics etc. Just purely mathematical result.
Thanks for any input.
The more the number of variables we include in the model, the seemingly better the fit and higher the $R^2$. Yes it is true. Every time you add a predictor to a model, the R-squared increases, even if due to chance alone. It never decreases. Consequently, a model with more terms may appear to have a better fit simply because it has more terms.But for this reason, we are supposed to interpret adjusted $R^2$.
The adjusted R-squared is a modified version of R-squared that has been adjusted for the number of predictors in the model. The adjusted R-squared increases only if the new term improves the model more than would be expected by chance. It decreases when a predictor improves the model by less than expected by chance.