If $E(Y\mid X)=a+bX$, show that $b =\frac{\mathrm{Cov}(X,Y)}{\mathrm{Var}(X)}$ where $a$ and $b$ are constants.
This question was asked before:
Proving $a$ (in $Y = aX + b + e$) satisfies $a = Cov(X, Y )/Var(X)$
But can someone suggest an answer without assuming bivariate normal distribution?
On
Least squares method. Consider the squared residual that must be minimized: $$R^2=\sum_{i=1}^n (y_i-ax_i-b)^2;\\ \begin{cases}(R^2)_a=-2\sum_{i=1}^n(y_i-ax_i-b)x_i=0\\ (R^2)_b=-\sum_{i=1}^n (y_i-ax_i-b)=0 \end{cases} \Rightarrow \\ \begin{cases}\color{red}{\sum_{i=1}^n x_i^2}\cdot a+\color{blue}{\sum_{i=1}^n x_i}\cdot b= \color{green}{\sum_{i=1}^n x_iy_i}\\ \color{red}{\sum_{i=1}^n x_i}\cdot a+\qquad \ \ \ \ \color{blue}n\cdot b=\color{green}{\sum_{i=1}^n y_i}\end{cases} \qquad \stackrel{Cramer}\Rightarrow \\ a=\frac{\begin{vmatrix}\color{green}{\sum_{i=1}^n x_iy_i}&\color{blue}{\sum_{i=1}^n x_i}\\ \color{green}{\sum_{i=1}^n y_i}&\color{blue}{n}\end{vmatrix}}{\begin{vmatrix}\color{red}{\sum_{i=1}^n x_i^2}&\color{blue}{\sum_{i=1}^n x_i}\\ \color{red}{\sum_{i=1}^n x_i}&\color{blue}n\end{vmatrix}}= \frac{n\sum_{i=1}^n x_iy_i-\sum_{i=1}^n x_i\cdot \sum_{i=1}^n y_i}{n\sum_{i=1}^n x_i^2-(\sum_{i=1}^n x_i)^2}=\\ \frac{\frac{\sum_{i=1}^n x_iy_i}{n}-\frac{\sum_{i=1}^n x_i}{n}\cdot \frac{\sum_{i=1}^n y_i}{n}}{\frac{\sum_{i=1}^n x_i^2}{n}-\left(\frac{\sum_{i=1}^n x_i}{n}\right)^2}=\\ \frac{Cov(X,Y)}{Var(X)}.$$
Observe that: $$\mathbb{E}Y=\mathbb{E}\left[\mathbb{E}\left[Y\mid X\right]\right]=\mathbb{E}\left[a+bX\right]=a+b\mathbb{E}X$$ so that:
$$\mathsf{Cov}\left(X,Y\right)=\mathbb{E}\left[\left(X-\mathbb{E}X\right)\left(Y-\mathbb{E}Y\right)\right]=\mathbb{E}\left[\mathbb{E}\left[\left(X-\mathbb{E}X\right)\left(Y-\mathbb{E}Y\right)\mid X\right]\right]=$$$$\mathbb{E}\left[\left(X-\mathbb{E}X\right)\mathbb{E}\left[\left(Y-\mathbb{E}Y\right)\mid X\right]\right]=\mathbb{E}\left[\left(X-\mathbb{E}X\right)\left(a+bX-\mathbb{E}Y\right)\right]=b\mathbb{E}\left(X-\mathbb{E}X\right)^{2}=b\mathsf{Var}X$$