Consider a binary classification with one binary output $y$ and two binary features $x_1$ and $x_2$. The Naive Bayes classifier assumes the following distribution for a pair: $$ p(y,x_1, x_2) = p(x_1|y) p(x_2|y) p(y) $$ Let the values of the probabilities be \begin{align*} p(y = 0) &= 0.5\\ p(x_1 = 1|y = 0) &= 0.9 \\ p(x_2 = 1|y = 0) &= 0.5 \end{align*}
\begin{align*} p(y = 1) &= 0.5\\ p(x_1 = 1|y = 1) &= 0.2 \\ p(x_2 = 1|y = 1) &= 0.5 \end{align*} Assumption: the features $x_i$ are assumed to be conditionally independent given the target $y$. For example $$ p(x_1|y,x_2) = p(x_1|y) $$ Compute the posterior values: $$ p(y = 1|x_1 = 1, x_2 = 1) \quad \text{and} \quad p(y = 0|x_1 = 1, x_2 = 1) $$ How would you classify an exmple with $x_1 = 1$ and $x_2 = 1$ \newline \newline \textbf{solution}: Using Bayes rule we have: \begin{align*} p(y = 1|x_1 = 1, x_2 = 1) &= \frac{p(x_1 = 1, x_2 = 1|y = 1)p(y = 1)}{p(x_1 = 1, x_2 = 1)}\\ &\propto p(x_1 = 1, x_2 = 1|y = 1)p(y = 1)\\ &= 0.2\cdot 0.5 \cdot 0.5\\ &= 0.05\\ &\text{}\\ &\text{and}\\ &\text{}\\ p(y = 0|x_1 = 1, x_2 = 1) &= \frac{p(x_1 = 1, x_2 = 1|y = 0)p(y = 1)}{p(x_1 = 1, x_2 = 1)}\\ &\propto p(x_1 = 1, x_2 = 1|y = 0)p(y = 0)\\ &= 0.9\cdot 0.5 \cdot 0.5\\ &= 0.225 \end{align*} Therefore we get $$ p(y = 1|x_1 = 1, x_2 = 1) = 0.2/1.1 \approx 0.18 \quad \textbf{(1)}$$ The example should be classified as $y = 0$
Here is what I don't understand: how do you get both $0.2$ and $1.1$ in (1) with the given information to be able to compute the result of (1)?
I do not know where those numbers come from but here is how I would do the computation. $$ P(x_1=1,x_2=1|y=1)=P(x_1=1|y=1)P(x_2=1|y=1)=0.2\cdot 0.5=0.1 $$ (by conditional independence). Same with $y=0$: $$ P(x_1=1,x_2=1|y=0)=P(x_1=1|y=0)P(x_2=1|y=0)=0.9\cdot 0.5=0.45 $$ Next,to be able to use Bayes rule, we need the total probability $$ P(x_1=1,x_2=1)=P(x_1=1,x_2=1|y=1)P(y=1)+P(x_1=1,x_2=1|y=0)P(y=0)=0.1\cdot0.5+0.45\cdot 0.5=0.275 $$ Using Bayes rule now, $$ P(y=1|x_1=1,x_2=1)=\frac{P(x_1=1,x_2=1|y=1)P(y=1)}{P(x_1=1,x_2=1)}=\frac{0.1\cdot 0.5}{0.275}=0.18. $$