As I understand the Variance-Bias tradeoff, modifying estimators to minimise bias might increase the variance of the estimator and vice-versa. For the simple case of the biased variance estimator, using $\frac{n}{(n-1)}$ as a correction factor might overcome the bias but the estimator has an un-optimal variance.
At several places regarding this, as a more suited goal of supervised learning, correction factors to minimise the Mean Squared Error are used ($\frac{n}{n+1}$ for the sample variance). Why is minimising MSE a better objective (than that of minimising only bias)?
Unbiasedness by itself doesn't amount to much. Suppose $X_1,\ldots,X_n \sim \text{i.i.d. Bernoulli} (p)$, i.e.
$$X_i = \begin{cases} 1 & \text{with probability } p, \\ 0 & \text{with probability } 1-p. \end{cases}$$
Then $X_1$ is an unbiased estimator of $p$, but it's a lousy estimator. On the other hand $(X_1+\cdots+X_n)/n$ is a far better unbiased estimator, since it is highly probable that it is close to $p$ because its variance is small.
In some settings, every unbiased estimator is far worse than many biased estimators. Here is my own example of that: http://arxiv.org/pdf/math/0206006.pdf
For the record, let us recall an identity: $$ \text{mean squared error} = \text{variance} + (\text{bias})^2. $$