I'm attempting the following questions
The first question is fine. However i feel like things get interesting in the second question. Now i'm not too familiar with things like functionals but this kind of approach made sense to me the most because i feel as simplified solutions i have found overlook a subtlety and just say c) follows from b) by essentially noticing $\mathbb{E}(\mathbb{E}(Y|X)) = \mathbb{E}(Y)$. As far as i'm aware $\mathbb{E}((Y-f(X))^2|X)$ returns a random variable and not a value in $\mathbb{R}$. Where as $\mathbb{E}(Y-f(X))^2$ returns a value in $\mathbb{R}$. If this is correct then what does it mean to minimise the random variable in b. ?
Well we can deduce that $\mathbb{E}((Y-f(x))^2|X) = (f(x) - \mathbb{E}(Y|X))^2 - (\mathbb{E}(Y|X))^2 + E(Y^2|X)$. Now if we let $\mathcal{F}$ denote the space of functions we want $f' \in \mathcal{F}$ that satisfies
$(f'(x) - \mathbb{E}(Y|X))^2 - (\mathbb{E}(Y|X))^2 + E(Y^2|X) \leq (f(x) - \mathbb{E}(Y|X))^2 - (\mathbb{E}(Y|X))^2 + E(Y^2|X)$ for all $f \in \mathcal{F}$. Now clearly $f'(x) = \mathbb{E}(Y|X)$ satisfies this as the square term is then $0$.
$\textbf{Key Result}$: So let the random variable $F' =(f'(x) - \mathbb{E}(Y|X))^2 - (\mathbb{E}(Y|X))^2 + E(Y^2|X)$ where $f'(x) = \mathbb{E}(Y|X)$ and let the random variable $F = (f(x) - \mathbb{E}(Y|X))^2 - (\mathbb{E}(Y|X))^2 + E(Y^2|X)$ where $f$ is any function in $\mathcal{F}$ then i know it's always the case that $F' \leq F$
Now i can use the key result for part c. I know that $\mathbb{E}(Y-f(x))^2 = \mathbb{E}(\mathbb{E}((Y-f(x))|X)$. For two random variables $X, Y$ its trivial that if $X \leq Y \Rightarrow \mathbb{E}(X) \leq \mathbb{E}(Y)$. So again considering the space of functions $\mathcal{F}$. Let $f' = \mathbb{E}(Y|X)$ and obtain $F'$ from $f'$ and $F$ from any other $f$ like we did before. Then i know that $F' \leq F$ and thus $\mathbb{E}(F') \leq \mathbb{E}(F)$. So i can conclude that to minimize $(Y- f(X))^2$ we require $f(x) = \mathbb{E}(Y|X)$.
Firstly is my reasoning correct? If it is correct have i over complicated it and is there a simpler solution? Thanks.
The intuition is that $E[(Y-c)^2|X=x]$ is minimized over all $c \in \mathbb{R}$ in the same way as in part (a), except now we condition on a world in which $X=x$.
Another approach is to note \begin{align} (Y-f(X))^2 &= (Y-E[Y|X])^2 + (E[Y|X]-f(X))^2 \\ & \quad + \underbrace{2(Y-E[Y|X])(E[Y|X]-f(X))}_{\mbox{operate}} \end{align} and take conditional expectations given $X$, operate on the third term.