The predictive distribution

34 Views Asked by At

I'm currently studying a bit of Bayesian statistics, and I got my self a bit stuck on the following result regarding the predictive distribution for $\mathbf{Y}|\mathbf{X = x}$.

My book states $$ f(\mathbf{y}|\mathbf{x}) = \int_{\theta\in\Omega} f(\mathbf{y},\theta|\mathbf{x})d\theta = \int_{\theta\in\Omega} f(\mathbf{y}|\theta, \mathbf{x}) f(\theta|\mathbf{x})d\theta = \int_{\theta\in\Omega} f(\mathbf{y}|\theta) f(\theta|\mathbf{x})d\theta $$

I'm unsure how we get $$ f(\mathbf{y},\theta|\mathbf{x}) = f(\mathbf{y}|\theta, \mathbf{x}) f(\theta|\mathbf{x}) $$ I've attempted to use $$ f(\mathbf{x_a},\mathbf{x_b}) = f_{\mathbf{x_b}|\mathbf{x_a}}(\mathbf{x_b}|\mathbf{x_a})f_\mathbf{x_a}(\mathbf{x_a}) $$ but unsuccessfully. I've also tried $$ f(\mathbf{y},\theta) = f_{\mathbf{Y|\theta}}(\mathbf{y}|\theta)f(\theta) $$ but in unsure how to make it all conditional on $\mathbf{X}$. So yeah, I'm tying knots on my self a bit. Additionally, I'm also unsure about the step $f(\mathbf{y}|\theta, \mathbf{x}) = f(\mathbf{y}|\theta)$. I'm guessing this is because $\mathbf{Y}$ and $\mathbf{X}$ are independent. So, we have $f(\mathbf{y}|\theta, \mathbf{x}) = f(\mathbf{y}|\theta)f(\mathbf{x})$, why why is $f(\mathbf{x}) = 1$?

I'm sure this is quite straightforward, since they offered no explanation of this in the book, and I'm probably just confusing my self a bit. But, I lost the better part of an hour trying on it already, and I think I might be fixating on a wrong approach.

1

There are 1 best solutions below

1
On BEST ANSWER

$f(y, \theta \mid x) = f(y \mid \theta, x) f(\theta, x)$ is a special case of $f(y,\theta) = f(y \mid \theta) f(\theta)$ where everything is conditional on $x$. If this is still not convincing to you, just go back to the original equation $f(y, \theta \mid x) = f(y \mid \theta, x) f(\theta, x)$ and plug in $f(y, \theta \mid x) = f(y,\theta,x) / f(x)$ and $f(y \mid \theta, x) = f(y,\theta,x)/f(\theta, x)$ to explicitly verify this.


$f(y \mid \theta, x) = f(y \mid \theta)$ is not true in general, but is true if $x$ and $y$ are conditionally independent given $\theta$. In fact this equation is sometimes taken as the definition of conditional independence; an equivalent definition is $f(y,x \mid \theta) = f(y \mid \theta) f(x \mid \theta)$.