I have a question regarding bagging linear models.
Suppose you wanna do linear regression on data X and y.
Alice directly implements (OLS) Linear Regression on it. The model is A1.
Bob applies bagging. More specifically, the sample is bootstrapped twice, and for each bootstrap sample, he runs an OLS Linear Regression on it and averages them. The model is A2.
The question is: compare the bias and variance between A1 and A2.
Now my friends have different ideas about that.
One opinion is that bagging will reduce the variance so A2 has a lower variance. Also, because A1 is BLUE and unbiased, it is impossible that A2 is unbiased. Therefore, A2 has a higher bias and lower variance.
The other opinion is that for every model of bootstrap since every sample has an equal opportunity to be selected, the small model should also be unbiased as A1. However, A1 is BLUE so A2 must have a higher variance. In conclusion, A2 has the same bias but higher variance.
Which opinion is correct? Why the other one is incorrect?
Thanks in advance.