Estimating posterior distribution when realization is unknown

Question

I'm working on a problem that can be reduced down the following scenario:

Consider two people, Alice and Bob. Alice has a prior probability distribution on a variable (which is known by Bob as well) while Bob knows the true probability distribution.

Alice privately draws a sample from the true (unknown, to her) distribution and updates her prior to a posterior factoring in the sampled value.

Since the sampling was private, Bob does not know what the sample was; however, Bob still wants to estimate what Alice's posterior is. What is this estimate?

My guess: It seems like Bob should consider every possible sample with its corresponding likelihood (from the true distribution) and use those to update the prior. So, in expectation, Bob's estimate of Alice's prior would just be the prior "sharpened" around the mean of the true probability distribution?

Question: How do I formally write this? Does it make sense to write the "expected" posterior given a collection of events (samples)?

Answer 1

I do not think that quite works in the way you describe it. This could get quite complicated as Alice's prior could be over various possible distributions and not necessarily over a single parameter

But for a simple example let's suppose we have a Bernoulli random variable $X$ with $\mathbb P(X=1)=p$ and $\mathbb P(X=1)=1-p$, and Bob knows that $p=0.7$ while Alice does not and instead Alice's prior for $p$ is uniform on $[0,1]$ updated after a single observation. This diagram shows what happens to the densities:

• Alice's prior density is the horizontal black line
• With an observation of $1$ she has a posterior density like the upward sloping red line and Bob knows this has probability $0.7$
• With an observation of $0$ she has a posterior density like the downward sloping pink line and Bob knows this has probability $0.3$
• So Bob could take a weighted average of Alice's two possible posterior densities to give the blue line

I do not think Bob's weighted average of Alice's posteriors can quite be described as just being the prior "sharpened" around the mean of the true probability distribution i.e. around $0.7$; the mode of the blue density is $1$, which is some way from $0.7$. What is true that the mean of the weighted average posterior of about $0.5667$ is closer to the true value of $0.7$ than the mean of $0.5$ of the prior was, but that is a rather weaker statement and would get even more convoluted if Alice's prior was over distributions rather over a single value