I was going through Chapter 3 (page 63 bottom) of Elements of Statistical Learning. While explaining regularization in ridge regression authors make the following statements.
"When there are many correlated variables in a linear regression model their coefficients can be poorly determined and exhibit high variance. A wildly large positive coefficient on one variable can be canceled by a similarly large negative coefficient on it correlated cousin"
It is clear that regularization shrinks the parameters. But I am unable to clearly understand its relation to correlation.
When variables are highly correlated, the parameter of a single(multiple also) predictor can go very large and with compensating with negative coefficients on correlated counterparts. This can lead to very high variance among parameters.