Puzzle $1$: A coin $C_1$ has probability $p$ of turning up head, while a coin $C_2$ has probability $2p$ of turning up head. All we know is that $0 < p < \frac12$. Now, $20$ tosses are given. You can choose all tosses from $C_1$, all tosses from $C_2$, or some tosses from each (the total is $20$). If the objective is to estimate $p$, what do you do? (Hint: Minimize mean-squared error of the estimator).
If we choose all $n = 20$ tosses from $C_1$, then we get $X_1,X_2,\ldots,X_n \sim \text{Ber}(p)$. $\overline{X_n}$ is an estimate for $p$ which is unbiased, and hence $$\text{MSE}_{\overline X_n}(p) = \text{var}(\overline X_n) = \frac{np(1-p)}{n^2} = \frac{p(1-p)}{n}$$ since $\text{MSE}_{\overline X_n}(p) = \text{var}(\overline X_n) + (\text{bias}(\overline X_n))^2$. Similarly, if we choose all tosses from $C_2$, we should get $$\text{MSE}_{\frac{\overline X_n}{2}}(p) = \text{var}\left(\frac{\overline X_n}{2}\right) = \frac{p(1-2p)}{2n}$$ right? The factor of two in the denominator is due to the fact that we're estimating $p$, while drawing from $\text{Ber}(2p)$.
What happens if we choose some from $C_1$ and some from $C_2$?
Answer: Choose all from $C_2$.
Here's another puzzle with a different answer:
Puzzle $2$: A factory produces light bulbs having an exponential distribution with mean $µ$. Another factory produces light bulbs having an exponential distribution with a mean of $2µ$. Your goal is to estimate $µ$. You are allowed to choose a total of $50$ light bulbs (all from the first, all from the second, or some from each factory). What do you do?
Answer: It does not matter what factory you choose from.
In this case, I saw that the mean-squared error if we choose all bulbs from the first factory is $\mu^2/n$, and it is the same if we choose all from the second factory as well. Due to the square of $\mu$ in the variance of this exponential distribution, it so happens that all choices make equivalently good estimators of $\mu$.
Follow-up question: Is there a way to probabilistically approach both these puzzles, i.e. without getting into actual MSE calculations to predict the MSE?