In this snip taken below from Bishop's PRML book, I'm having trouble following the derivation that allows us to conclude that the minimizing hypothesis function, labeled y(t) here, can be shown to be equal to $\mathbb{E}[$t|x].
Perhaps my issues is 1) lack of knowledge of the calculus of variations that allows for removing one of the integrals by taking a partial derivative with respect to y(x) in (1.88). And secondly, I'm not following what "sum of product rules of probability" are being used to derive (1.89).
Could anyone walk me though these last 2 lines in a more detailed step by step fashion?
$\def\qty#1{\left(#1\right)}$ This is much easier to understand if you avoid the drama of the calculus of variations. The expression in (1.87) is equivalent to, $$ \mathbb{E}[L]=\mathbb{E}_{x,t}\left[\qty{y(x)-t}^2\right] $$ where $\mathbb{E}_{x,t}$ is the expectation w.r.t. the joint distribution of the random variables $x$ and $t$. Now lets fix $x$ and write, $$ \mathbb{E}[L|x]=\mathbb{E}_{t}\left[\qty{y(x)-t}^2\right | x] $$ Because we have fixed $x$, $y(x)$ can be treated as a variable. This means that we can take the partial derivative w.r.t. $y(x)$, $$ \frac{\partial \mathbb{E}[L|x] }{\partial y(x)} = 2\mathbb{E}_{t}\left[\qty{y(x)-t}\right | x] $$ Set the r.h.s. to zero; the solution is, $$ y^*(x)=\mathbb{E}_{t}\left[t | x\right] $$ i.e. given $x$ the optimal value for $y$ is equal to the expectation of $t$ w.r.t. the conditional distribution $p(t|x)$ which is (1.89). So for each value of $x$ we have an optimal value for $y(x)$; since, $$ \mathbb{E}[L]=\mathbb{E}_{x}\left[\mathbb{E}_{t}\left[\qty{y(x)-t}^2\vert x\right] \right] $$ if we use $y^*(x)$ we have also optimised $\mathbb{E}[L]$.