(also posted on CV, but I will try here too) I am trying to find out if what I am looking at is a known problem. I am considering the case of weighted least squares, and I am trying to find the optimal weights of biased measurements. I have shown what the value of the weight must be to minimize the MSE of this WLS estimator. In other words, if we know the squared bias of some measurements, we can lower the MSE by including them in the parameter estimator, and to include them in the WLS, we need to compute a (optimal) weight. Is this a thing? Where can I read more about it? All hints are appreciated.
Problem:
We want to estimate $\beta$ in $$ y_i = \beta x_i + \epsilon_i $$ where $\epsilon_i$ are Gaussian, some with zero mean, some have nonzero mean, while $\beta, x_i$ are deterministic. We assume to know the squared bias of those $\epsilon_i$ that are biased (in practice this is estimated), and we also know the noise of the unbiased measurements.
I encountered this problem as I am working on a data fusion problem, where one information source will be biased. The linear case is just for completion, to look at optimums in ideal situations. (I have proved what the optimal weights are for the WLS both when this bias is deterministic and Gaussian distributed)
Side note: I keep reading about how MMSE estimator is unbiased, without the sources pointing to any assumption on the noise. Without assumptions on the noise (zero mean) it can not be true, right?