So I've been given this formula For regression $R^2=1 - \sum \frac{{(y_i - \hat{y}_i)}^2}{(y_1-\bar{y})^2}$
Now an obvious question that has come to me is why $R^2$ stays the same in certain scenarios, for example:
so the regression formulas are:
$$\mathrm{TESCO}= 0.2830 + 0.7160 \,\mathrm{FTSE100} $$
$R^2: 10.6% $
$$\mathrm{FTSE100} = 0.8530 + 0.1475 \, \mathrm{TESCO} $$
$R^2: 10.6% $
then why is the coefficient of determination is the same in both cases
Any explanation on this would be great, thank you!
$R^2$ is the square of the correlation between TESCO and FTSE100, so you can rely on the symmetry in the definition of correlation.
Your formula for $R^2$ ought to say $R^2 = 1-\dfrac{\sum(y_i-\hat y_i)^2}{\sum(y_i-\overline y)^2}$. So we have $$ R^2 = \frac{\sum(y_i-\overline y)^2- \sum(y_i-\hat y_i)^2}{\sum(y_i-\overline y)^2}.\tag 1 $$
Now \begin{align} & \sum\big((y_i-\hat y_i)+(\hat y_i - \bar y)\big)^2 \\[6pt] ={} & \left(\sum(y_i-\hat y_i)^2\right) + \underbrace{2\left(\sum(y_i-\hat y_i)(\hat y_i-\bar y)\right)}_{\text{This adds up to $0$.}} + \left(\sum(\hat y_i - \overline y)^2\right). \end{align} If the sum in the middle is indeed zero, then $(1)$ reduces to $$ R^2 = \frac{\sum(\hat y_i-\overline y)^2}{\sum(y_i-\overline y)^2}. \tag 2 $$ The least-squares line is $\hat y=\hat a+\hat b x$, so $(2)$ becomes \begin{align} & \frac{\sum(\hat b x_i - \hat b \overline x)(\hat y_i-\overline y)}{\sum(y_i-\overline y)^2} = \frac{ \hat b\sum(x_i - \overline x)(\hat y_i-\overline y)}{\sum(y_i-\overline y)^2} \\[8pt] = {} & \frac{ \sum(x_i - \overline x)(\hat y_i-\overline y)}{\sum(y_i-\overline y)^2}\cdot\frac{\hat b\sum(x_i-\overline x)^2}{\sum(x_i-\overline x)^2} \\[8pt] = {} & \frac{ \sum(x_i - \overline x)(\hat y_i-\overline y)}{\sum(y_i-\overline y)^2}\cdot\frac{\sum(x_i-\overline x)(y_i-\overline y)}{\sum(x_i-\overline x)^2} \\[8pt] & [\text{Next we would like to do the following;} \\ & {}\quad\text{more on this will appear further below.}] \\[8pt] = {} & \frac{\sum(x_i-\overline x)(y_i-\overline y)}{\sum(y_i-\overline y)^2} \cdot\frac{\sum(x_i-\overline x)(y_i-\overline y)}{\sum(x_i-\overline x)^2} = \frac{\left(\sum(x_i-\overline x)(y_i-\overline y)\right)^2}{\sum(y_i-\overline y)^2\sum(x_i-\overline x)^2}. \tag 3 \end{align}
So how do we know that $$ \sum(x_i - \overline x)(\hat y_i-\overline y) = \sum(x_i-\overline x)(y_i-\overline y)\text{ ?} $$ Here we only need the fact that $$ \sum (y_i-\hat y_i)(x_i-\overline x) = 0. $$ This says the vector of residuals is orthogonal to the vector of predictors. That is true because the vector of least-squares fitted values $\hat y_i$ is the orthogonal projection onto the vectors $(1)_{i=1}^n$ and $(x_i)_{i=1}^n$. This also explains why the "sum in the middle" above is $0$.
Now the punch line: Notice that the final expression in $(3)$ is symmetric in $x$ and $y$. In other words, it remains the same if the roles of $x$ and $y$ are interchanged!
(And it's the square of the correlation, but we don't need that here. This fact, however, explains why $R^2$ is called $R^2$.)