Mixing conditional probabilities with prior probabilities

208 Views Asked by At

Consider a finite set $S$, called the state space. Let $\Delta S$ be the set of all probability distributions on S. Consider a partition $\Pi$ of $S$, which is a collection of mutually disjoint subsets $E$ of $S$, whose union is $S$. Let $P \subseteq \Delta S$ be such that $p \in P$ implies $p(E)> 0$ for all $E \in \Pi$.

For each $E \in \Pi$ and $p \in P$, define $p_E = \frac{p}{p(E)}$ to be the Bayesian update of $p$ given $E$. Let $P_E$ be the set of all these conditionals of $P$ given $E$.

For each $r \in P$ and $p_E \in P_E$, define $p_E \otimes^E r \in \Delta S$ as follows: for each $F \subseteq S$, $p_E \otimes^E r (F) = r(E) p_E(F) + r(F \cap E^c)$. Hence, the choice of $p_E$ determines all probabilities given $E$, whereas $r$ determines all other probabilities.

Consider two properties:

a) Suppose convex set of probabilities P has the following property: For each $E \in \Pi$, for each $r \in P$ and each $p_E \in P_E$, we have $p_E \otimes^E r \in P$.

b) Suppose convex set P has the following property: it contains all probability measures that have the following property: take $r \in P$ and define $q = \underset{E \in \Pi}{\sum} r(E)p_E$, where $p_E \in P_E$. Then, $q \in P$. Note that $p_{E_1}$ and $p_{E_2}$ may not be conditionals of the same $p \in P$. Hence, $q$ is just the convex combination (with weights from $r$) of conditionals, one for each $E \in \Pi$.

The question is, if P satisfies property a), does it satisfy b)? And if it satisfies property b), does it satisfy property a)? Thanks!

1

There are 1 best solutions below

3
On

Properties a) and b) are equivalent.

Your overloaded notation in b), in which the subscript on $p_E$ appears to denote both an index $E$ and the conditionalization on $E$, makes it difficult to express things clearly, so I'll write this as $p^E_E$, where $p^E$ is a distribution indexed by $E$ and $p^E_E$ is that distribution conditional on $E$, consistent with the notation you introduce at the beginning.

That b) implies a) is straightforward: In b), choose $p^E=r$ for all $E\in\Pi$ except $E=G$; then

$$ \sum_{E\in\Pi}r(E)p^E_E(F)=r(G)p^G_G(F)+\sum_{E\ne G}r(E)r_E(F)=r(G)p^G_G(F)+r(F\cap G^c)\;, $$

so any operation in a) can be mimicked by a single operation in b). In the other direction, you need several steps, one for each $E\in\Pi$. To mimick $\sum_{E\in\Pi}r(E)p^E_E$, start out with $r$ and in each step replace it by $p^E_E \otimes^E r$. Doing this for all $E\in\Pi$ transforms $r$ stepwise into $\sum_{E\in\Pi}r(E)p^E_E$.