Given two random variables $X$ and $Y$, I want to find a function $f(\cdot)$ such that the correlation coefficient $Cor(Y,f(X)) = \frac{Cov(Y,f(X))}{Std(Y)Std(f(X))}$ is maximized. Intuitively, such $f$ should be the conditional expectation, i.e., $f(X) = \mathbb{E}[Y|X]$. I wonder is there a way to formally justify this? Or it's some known result?
PS: My question is very related to the control variates method to reduce variance in MC simulation.
It can. First, we can assume without loss of generality that $E(Y)=0$ and $V(Y)=1$. Also, the solution will not be unique: replacing $f$ by $\lambda f+\mu$ does not affect the correlation so we can assume $E[f(X)]=0$ and $V(f(X))=E[E(Y|X)^2]$. Then, we just need to maximize $E[Y f(X)]$. Now, $$E[Y f(X)] =E[E(Y|X) f(X)] \leq E[E(Y|X)^2]^{1/2} E[f^2(X)]^{1/2} = E[E(Y|X)^2].$$ The equality is reached by considering $f(X)=E(Y|X)$. The result follows.