Linear regression where the error is modified

65 Views Asked by At

I have a set of coordinates $\{(x_1,y_1),(x_2,y_2),\ldots,(x_n,y_n)\}$ , where for every $i<n$, $$ x_i \ll x_{i\space+\space1}\space\space\text{and}\space\space y_i\ll y_{i\space+\space1} $$ I know that the data points have a linear relationship $y=\alpha\space+\space\beta x$.

However, when using the simple slope formula for the regression $\beta=\frac{\sum_{i=1}^n(x_i-\overline{x})(y_i-\overline{y})}{\sum_{i=1}^n(x_i-\overline{x})^2}$, due to the nature of larger values, the entire formula simplifies to $\beta=\frac{y_n}{x_n}$, which ignores all data-points but the largest one. The regression line I need must take the other data-points (and the errors they contribute) into consideration.

One potential solution to this problem is looking at the errors in a way that the bigger they become, the less their impact increases. In other words, using the error equation $$ \ln\hat\varepsilon_i=y_i-\alpha-\beta x_i $$ Instead of the original equation $\hat\varepsilon_i=y_i-\alpha-\beta x_i$.

In the end, this all boils down to finding the ordered pair $(\hat\alpha,\hat\beta)$ which solves $$ \min_{\alpha,\space \beta} Q(\alpha,\beta)\space\space \text{Where}\space\space Q(\alpha,\beta)=\sum_{i=1}^n e^{2(y_i-\alpha-\beta x_i)} $$ So what pair $(\hat\alpha,\hat\beta)$ solves this ordeal?


EDIT: as @user121049 mentioned, the function I applied on the error causes $\alpha$ to approach towards $\infty$. Therefore, we need a function to apply on our error that still accomplishes the behavior described above. This means the function $f$ must accomplish:

  1. the range of $f$ must be $\Bbb{R}$ (this means the error can be any value)

  2. $\lim_{x\to0^\pm}f(x)$ cannot be $\pm\infty$ (this makes sure neither $\alpha$ nor $\beta$ have a reason to approach towards $\pm\infty$)

  3. $f=o(x)$ (this makes sure that the error's impact increases less the bigger its value is)

  4. $f$ must be an increasing function (this makes sure the error becomes more effective the bigger it is)

A potential solution for $f$ is $f(x)=\left\{^{\text-\sqrt {\text-x};\;x<0}_{\,\sqrt x\;;\;x\ge0}\right.$ . This time, the equation we need to solve is: $$ \min_{\alpha,\space \beta} Q(\alpha,\beta)\space\space \text{Where}\space\space Q(\alpha,\beta)=\sum_{i=1}^n (y_i-\alpha-\beta x_i)^4 $$ In this scenario, what pair $(\hat\alpha,\hat\beta)$ is the solution?

NOTE: The use of other potential functions is also welcomed.