Consider the set of probability density functions over the bias of a coin. Let the random variable $X$ be the coin's bias for heads, and let the random variable $Y_n$ be the number of heads in the first $n$ tosses. For a given non-empty interval $(\alpha, \beta)$ (with $\alpha > 0$ and $\beta < 1$), a given $n$, a given $y_n$, and a given $\theta \in (\alpha, \beta)$, is it always possible to find a probability density function $f_X(x)$ with the following properties:
- $E[X] \in (\alpha, \beta)$,
- $E[X \mid Y_n = y_n] = \theta$, and
- $\int_a^b f_X(x) \, dx > 0$ for every $a, b \in [0,1]$ with $a < b$.
I think the answer is yes, but I don't know how to prove it. Any help is much appreciated. Thank you in advance.
Yes, it is. Construct two piecewise constant densities, each with a spike within $(\alpha,\beta)$, one to the left and the other to the right of $\theta$, with a low constant value outside the spike. By concentrating enough probability in the spikes, you can enforce $E[X \mid Y_n = y_n] \lt \theta$ for one, $E[X \mid Y_n = y_n] \gt \theta$ for the other and $E[X]\in(α,β)$ for both. Then form a suitable affine combination. It will still have $E[X]\in(α,β)$, and you can choose the coefficients such that $E[X \mid Y_n = y_n] = \theta$.