How do you simplify or find the closed form of the following probability distribution?
$$ p(y|x,X,Y) = \int^{10}_{-10} \int^{10}_{-10} p(y|x,w_1,w_2)p(w_1)p(w_2)dw_1dw_2 $$
Where: $$ p(w_1) = Uniform[-10,10] \\ p(w_2) = Uniform[-10,10] \\ p(y|x,w_1,w_2) \sim Normal(f_w(x),1) $$
$f_{w}(x)$ is a neural network function using a pair of weights $w_1$ and $w_2$. It can be non-linear.
The bigger context is the follows: the $w_1$ and $w_2$ uniform distributions are actually posterior distributions, so the formula is the predictive posterior distribution for $p(x|y,X,Y)$. The homework question asks me to compare this approach with an alternative approach where I first take the expectation of the posteriors - in this case 0 for both $w_1$ and $w_2$ and then use it to estimate p(y|x), in which case we get a Normal with mean equal to output of neural network using 0 weights. The question asks us to compare between the 2 methods. Do you have any opinions?
The full problem is in the following link: (problem 4) https://d1b10bmlvqabco.cloudfront.net/attach/jrk8b9xgnwd6se/j4y86yrf1e2490/jtcbx0p946j2/hw2.pdf.
It is not clear that $P(w_1)$ and $P(w_2)$ is the posterior in the assignment PDF, but I confirmed with my TA.
Thanks!
$$ p(y|x,X,Y) = \int^{10}_{-10} \int^{10}_{-10} p(y|x,w_1,w_2)\frac{1}{400}dw_1dw_2 $$
for starters using the uniform PDFs there. Now in $p(y|x)$ you will have a function of the form $e^{-(f_\omega(x)-y))^2/2}$. This will integrate differently depending on your $f(\omega;x)$.