I need to fit data to a line. This data also includes tolerances.
Now, I was wondering, if there exists a method, to account for those tolerances?
For instance, a data point with a lot of tolerances should not be given the same weight as a data point with small tolerances.
Here is a sample of my data:
x| Data-point (y) | +/- tolerance
1| 120.984 | 0.426248
2| 132.194 | 0.418291
3| 130.413 | 1.260100
4| 127.687 | 0.414457
5| 125.710 | 0.410716
6| 128.188 | 1.431200
This is mostly just hashing out some details from the comments section above.
We are given with a model matrix $A$ (which in this case has a column of $1$s and a column of $x$ values), and a vector of true $y$ values $y$. If we choose a parameter vector $p$ (in this case containing an intercept and a slope), we have the predicted $y$ values $\hat{y}=Ap$. In standard least squares we try to minimize $\| \hat{y}-y \|^2$, where $\| \cdot \|$ is the Euclidean norm.
In weighted least squares, we weigh some entries more and others less. Given standard deviations $\sigma_i$ for the measured $y_i$, it makes sense to think of $(\hat{y}-y)_i$ in units of $\sigma_i$. We can achieve that by making a diagonal matrix $\Sigma$ with diagonal entries $\sigma_i$ and then trying to minimize $\| \Sigma^{-1}(Ap-y) \|^2$. We'll rewrite this as $\langle Ap-y,Ap-y \rangle$ where $\langle x,y \rangle$ is defined to be $(x,\Sigma^{-2}y)$, where $(\cdot,\cdot)$ is the Euclidean inner product.
Then we require the solution $p^*$ to be such that $Ap^*-y$ is orthogonal to all vectors of the form $Ap$ in our weighted inner product. That is $\langle Ap,Ap^*-y \rangle = 0$ or $p^T A^T \Sigma^{-2}(Ap^*-y)=0$ for every $p$. This occurs if and only if $A^T \Sigma^{-2}(Ap^*-y)=0$ (just check it with $p$ equal to the standard basis vectors). Thus the normal equations in the end read
$$A^T \Sigma^{-2} A p^* = A^T \Sigma^{-2} y.$$