Linear Regression:
$$Y=a+bX + \epsilon$$
Hi. For R squared in linear regression, I saw them in the form of ratio between $(y_i-y^{bar})$, or in terms of $$(S_{xy})^2/(S_{xx}S_{yy})$$I'm not sure if you guys come across this form:
$$R^2=\frac{Var(bX)}{V(bX) +V(\epsilon)}$$ ?
Assume that the process that generates the i.i.d data $\{(x_i, y_i\}_{i=1}^n$ is $ y_i = a + bx_i + \epsilon_i $, where $E \epsilon_i = 0$ and $E\epsilon_i^2 = \sigma^2 < \infty$. Now, recall that $$ R^2 = \frac{\sum ( \hat{y}_i - \bar{y})^2}{\sum(y_i - \bar{y})^2} = 1 - \frac{\sum(\hat{y}_i - y_i)^2}{\sum(y_i - \bar{y})^2} = 1-\frac{\sum(\hat{y}_i - y_i)^2/n}{\sum(y_i - \bar{y})^2/n} = 1-\frac{ \hat{\sigma}^2_{\epsilon} }{\hat{\sigma}^2_Y}. $$ Namely, the sample-based measure $R^2$ is a biased estimator of a population parameter that is $$ \rho = 1-\frac{\sigma^2_{\epsilon}}{\sigma^2_Y}. $$ So, $$ \rho= 1 - \frac{Var(\epsilon_i)}{Var(y_i)} = 1-\frac{Var(\epsilon_i)}{Var(bx_i)+Var(\epsilon_i)} = \frac{Var(bx_i)}{Var(bx_i)+Var(\epsilon_i)}. $$