Posterior Probability Density for a set of random variables

38 Views Asked by At

I've been reading Watanabe's book on Bayesian Statistics, and had trouble with the posterior probability density formula for a set of random variable pairs.

For a set of random variables $X^n=(X_1,X_2,...,X_n)$ subject to $q(x_1)q(x_2)...q(x_n)$ i.e. independent random variables (each of which are $\Bbb R^N$), the posterior of the model parameter $w \in W \subseteq \Bbb R^d$ is a straightforward application of the Bayes rule (where the prior represents $\varphi(w)$):

$$ p\left(w \mid X^n\right)=\frac{1}{Z\left(X^n\right)} \varphi(w) \prod_{i=1}^n p\left(X_i \mid w\right) $$

$$ Z\left(X^n\right)=\int \varphi(w) \prod_{i=1}^n p\left(X_i \mid w\right) d w $$

However, Watanabe at (1.9) then claims, for $(X^n,Y^n) = ((X_1,Y_1),(X_2,Y_2),...,(X_n,Y_n))$ subject to $q(x_1,y_1)q(x_2,y_2)...q(x_n,y_n)$ i.e. independent random variable pairs (each sample on $\Bbb R^m \times \Bbb R^n$), the following is true:

$$ p\left(w \mid X^n, Y^n\right)=\frac{1}{Z\left(X^n, Y^n\right)} \varphi(w) \prod_{i=1}^n p\left(Y_i \mid X_i, w\right) $$

$$ Z\left(X^n, Y^n\right)=\int \varphi(w) \prod_{i=1}^n p\left(Y_i \mid X_i, w\right) d w $$

But this doesn't make sense; it seems to ignores $P(X_i \mid w)$ for the factorization of $P(X_i, Y_i)$ within $\prod$, nor do they seem to cancel out (since one's within the integral at the denominator). Am I missing something?

1

There are 1 best solutions below

2
On BEST ANSWER

I don't think that you are missing anything. In fact, you are totally right in pointing this out! There seem to be some assumptions hidden in the notation, in particular, in the formulation "for an arbitrary pair $(p(y|x,w),\varphi(w))$". I suppose that it might be implied that $X^{n}$ and $w$ are independent (and also that $X_{i}$ and $Y_{i}$ are conditionally independent given $w$), but, in my humble opinion, it is a somewhat counterintuitive introduction to posterior distributions.