Calculate decision boundary of two Gaussians with different missclassification costs

295 Views Asked by At

Assuming we have two classes $C_1$ and $C_2$ represented as two Gaussians with $(2\mu_2, \sigma)$ and $(\mu_2, \sigma)$. We know further that $\mu_2 > 0$ and $p(C_1) = p(C_2)$.

We want now calculate the decision boundary assuming that classifying $x \in C_2$ as $C_1$ is three times more expensive than the opposite. Classifying $x$ correctly has no costs.

At first we note that we need some representation of the loss function. Let $\alpha_1$ be the decision for $C_1$ and $\alpha_2$ the decision for $C_2$, then we get

$\begin{align} R(\alpha_1|x) &= \lambda_{11}p(C_1|x) + \lambda_{12}p(C_2|x)\\ R(\alpha_2|x) &= \lambda_{21}p(C_1|x) + \lambda_{22}p(C_2|x) \end{align}$

where $\lambda_{ij} = \lambda(\alpha_i|C_j)$ is the cost for deciding for class $i$ and being in fact class $j$.

Hence we have $\lambda_{11} = \lambda_{22} = 0$ and with $\lambda_{21} = c$ we get $\lambda_{12} = 3c$.

We are now searching the the decision boundary, where it holds $R(\alpha_1|x) = R(\alpha_2|x)$. We can use the previous equations and substitute the $\lambda$'s: \begin{align} R(\alpha_1|x) = 3c\cdot p(C_2|x) &= c \cdot p(C_1|x) = R(\alpha_2|x)\\ \Longleftrightarrow 3c\frac{p(x|C_2)p(C_2)}{p(x)} &= c\frac{p(x|C_1)p(C_1)}{p(x)}\\ \Longleftrightarrow 3p(x|C_2) &= p(x|C_1)\\ \Longleftrightarrow 3 \exp(-\frac{(x - 2\mu_2)^2}{2\sigma^2}) &= \exp(-\frac{(x - \mu_2)^2}{2\sigma^2})\\ \Longleftrightarrow \frac{\exp(-\frac{(x - \mu_2)^2}{2\sigma^2})}{\exp(-\frac{(x - 2\mu_2)^2}{2\sigma^2})} &= 3\\ \Longleftrightarrow - \frac{(x - \mu_2)^2}{2\sigma^2} + \frac{(x - 2\mu_2)^2}{2\sigma^2} &= \ln(3)\\ \Longleftrightarrow \frac{-x^2 + 2x\mu_2 - \mu_2^2 + x^2 - 4x\mu_2 + \mu_2^2}{2\sigma^2} &= \ln(3)\\ \Longleftrightarrow \frac{-2x\mu_2}{2\sigma^2} &= \ln(3)\\ \Longleftrightarrow x &= -\ln(3)\frac{\sigma^2}{\mu_2} \end{align}

Sorry for this long equivalent transformation. Is my basic approach correct? And is my result also correct?

Thank you for reading and helping!

1

There are 1 best solutions below

0
On BEST ANSWER

Your approach is correct. There is only a calculation mistake in one of the steps:

$$- \dfrac{(x - \mu_2)^2}{2\sigma^2} + \dfrac{(x - 2\mu_2)^2}{2\sigma^2} = \ln(3)$$

$$\iff \dfrac{-x^2 + 2x\mu_2 - \mu_2^2 + x^2 - 4x\mu_2 + 4\mu_2^2}{2\sigma^2} = \ln(3)$$

$$\iff \dfrac{3\mu_2^2-2x\mu_2}{2\sigma^2}=\ln (3)$$

$$\iff x=\dfrac{3\mu_2^2-2\sigma^2\ln(3)}{2\mu_2}.$$