I am trying to solve exercise 3.12 from Kevin P.Murphy-Machine Learning_ A Probabilistic Perspective
According to the question for a Bernoulli estimation the prior is
$$ p(\theta) = 0.5 \ => \ when\ \theta=0.5 $$ $$ p(\theta) = 0.5 \ => \ when\ \theta=0.4 $$ $$ p(\theta) = 0 \ => \ everywhere\ else $$
I need to calculate the Maximum A Posteriori Probability (MAP) under this prior.
I am unable to understand how to begin:-
a. Should I treat the prior as a Beta function and try and find x and y for the beta function using the given conditions.
b. Should I try to create a prior function that map the given conditions - if yes then what function can I use???
You are already given the prior in the problem description. It's not defined by a Beta distribution - it's the piecewise function
\begin{equation} p(\theta) = \left\{ \begin{array}{ll} .5 & \mbox{if $\theta = .4 \text{ or } .5$} \\ 0 & \mbox{else} \end{array} \right. \end{equation}
Now, we know that the posterior is proportional to the likelihood times the prior. Let $N_1$ represent the number of heads and $N_0$ represent the number of tails. Define $N = N_0 + N_1$.
\begin{align*} p(\theta|D) &\propto p(D|\theta) \cdot p(\theta) \\ &\propto \theta^{N_1}(1 - \theta)^{N_0} \cdot \left\{ \begin{array}{ll} .5 & \mbox{if $\theta = .4 \text{ or } .5$} \\ 0 & \mbox{else} \end{array} \right. \\ &= \left\{ \begin{array}{ll} \theta^{N_1}(1 - \theta)^{N - N_1} & \mbox{if $\theta \in \{.4, .5\}$} \\ 0 & \mbox{else} \end{array} \right. \\ &= \left\{ \begin{array}{ll} .4^{N_1}(.6)^{N - N_1} & \mbox{if $\theta = .4$} \\ .5^N & \mbox{if $\theta = .5$} \\ 0 & \mbox{else} \end{array} \right. \\ \end{align*}
Now we must choose $\theta$ to maximize the posterior.
$\hat{\theta}_{MAP} = \underset{\theta}{\mathrm{argmax}} p(\theta|D)$
It's obvious that $\hat{\theta}_{MAP} \in \left\{.4,.5\right\}$, so \begin{align*} \hat{\theta}_{MAP} &= \underset{\theta}{\mathrm{argmax}} \left\{ \begin{array}{ll} .4^{N_1}(.6)^{N - N_1} & \mbox{if $\theta = .4$} \\ .5^N & \mbox{if $\theta = .5$} \end{array} \right. \\ &= \left\{ \begin{array}{ll} .4 & \mbox{if $(.4)^{N_1} (.6)^{N-N_1} \ge .5^N$} \\ .5 & \mbox{else} \end{array} \right. \end{align*}