This is a rather conceptual question. I came across the following sentence in a text book:
"As with full Bayesian inference, MAP Bayesian inference has the advantage of leveraging information that is brought by the prior and cannot be found in the training data. This additional information helps to reduce the variance in the MAP point estimate (in comparison to the ML estimate). However, it does so at the price of increased bias."
This was stated in the standard context of using an MAP estimate in case of a linear regression, where this then corresponds to a regularization with a prior that has mean zero and variance 1/lambda * Identity.
I don't understand the last sentence. If we had chosen a prior that has its mean centred around the true values of the parameters that we are seeking to estimate (e.g. the weights in the example of linear regression), wouldn't the prior reduce the bias (or at least not increase it), because it would make the posterior distribution more peaked around the true parameters? Or is my thinking about the parameters confusing sth with the frequentist perspective?
Thanks!
Best, JZ