how to apply weighting factor to linear regression

Question

Say if I have two sets of data, x and y. And I am required to apply a weighting factor,1/x, to the regression line. Does that mean I should plot 1/y versus 1/x and then get the regression? Could anyone help me with this please? Thanks.

Answer 1

When you perform, say, a linear regression $y=a+bx$ based on the data points ($x_i,y_i$), if the $y_i$'s cover a large range, the largest values strongly contribute to define the parameters.

So, instead of minimizing $$SSQ=\sum_{i=1}^n (a+bx_i-y_i)^2$$ weighted least-square method minimize $$SSQ=\sum_{i=1}^n w_i(a+bx_i-y_i)^2$$ where $w_i$ is a weight assigned to each data point. In your case, $w_i=\frac 1 {x_i}$.

If you were using $w_i=\frac 1 {y_i^2}$, it would mean that you try to minimize the sum of squares of relative errors.

I suppose that what you are asked is to derive the formulae for $a,b$ when weights $w_i$ are used. Then, just write down the normal equations for $$SSQ=\sum_{i=1}^n \frac 1{x_i}(a+bx_i-y_i)^2$$ that is to say, compute $\frac{dSSQ}{da}$ and $\frac{dSSQ}{db}$; write that they are equal to zero. This will give two linear equations for the two unknowns $a$ and $b$.