Linear Regression Coefficients

Question

Suppose you have a OLS linear regression with $\beta_0, \beta_1$, you data is {$x,y$}.

Now, suppose we swap the labels of your data. $x\rightarrow y, y\rightarrow x.$

You get new $\beta':$ $\beta_0', \beta_1'$.

Here is the problem. What can you say about $\beta/\beta'?$ (for example, can it be 10?)

---

My attempt:

Let's first focus on $\beta_1$.

In OLS $\beta_1=S_{XY}/S_{XX}$, $\beta_1' = S_{YX}/S_{YY}$.

Thus, $\beta_1/\beta_1' = S_{YY}/S_{XX}$.

But the answer is that this ratio should be between 0 and 1. I got stuck.

$S_{YY}/S_{XX} = \frac{\sum(Y_i-Y^{bar})^2}{\sum(X_i-X^{bar})^2}$

Answer 1

The claim is wrong.

Regression coefficients have the following property:

$$r^2= \beta_{yx} \cdot \beta_{xy},$$ where $r$ is the correlation between x and y.

The product of the regression coefficients lies between 0 and 1 as $0\leq r^2\leq 1$.

Hence, it can be concluded that the ratio of the regression coefficients cannot lie between 0 and 1.

Answer 2

Code for an R experiment is below.

x <- rnorm(100, mean = 0, sd = 2)
y <- rnorm(100, mean = 0, sd = 1)
z <- rnorm(100, mean = 0, sd = 0.5)
coef(lm(y~x))[2]/coef(lm(x~y))[2] #slope when y is response/ slope when x is response
coef(lm(y~z))[2]/coef(lm(z~y))[2] #slope when y is response/ slope when z is response
coef(lm(z~x))[2]/coef(lm(x~z))[2] #slope when z is response/ slope when x is response
#  results 
#  0.3002797 
#  5.583115  
#  0.05378355 

Values for the ratios of $\hat \beta_1$ swapping response and predictor variables are: 0.3003, 5.5831 and 0.0538. So they don't have to fall in between 0 and 1, but they could.

Without further constraints/specifications, the best one can say is along the lines of the statement the OP found. OP states that the ratio will be equal to the ratio of the standard deviation of the response to the standard deviation of the predictor. It's close - instead it is the ratio of the sample variances, as noted in @Not_Dustin's comments.

One can see this from the formula $\hat \beta_1 = r \, \dfrac{S_y}{S_x}$.

So,
$$\frac{\hat \beta_1}{\hat \beta'_1} = \frac{r \, \dfrac{S_y}{S_x}}{r \, \dfrac{S_x}{S_y}} = \dfrac{S^2_y}{S^2_x}.$$