Consider the problem of finding the best line separating two sets of points, where each set of points is produced by a bivariate gaussian.
Please, is there a closed form solution to linear discriminant analysis in this setting?
Alternatively, are there other classifiers for which closed form solutions are available in this setting (e.g., likelihood ratio test)?
We would already be interested in a solution under the special case wherein the two given covariance matrices are the same.
The PDF is
$$f(x|C_i)=(2\pi)^{-1}\det(\Sigma_i)^{-\frac12}\exp\left(-\frac12 (x-\mu_{i})^T\Sigma_i^{-1}(x-\mu_i)\right)$$
Hence we have
$$f(C_i|x)=\frac{f(C_i)f(x|C_i)}{f(C_1)f(x|C_1)+f(C_2)f(x|C_2)}$$
At the bounday, we have
$$f(C_1)f(x|C_1)=f(C_2)f(x|C_2)$$
$$f(C_1)\det(\Sigma_1)^{-\frac12}\exp\left(-\frac12 (x-\mu_{1})^T\Sigma_i^{-1}(x-\mu_1)\right)=f(C_2)\det(\Sigma_2)^{-\frac12}\exp\left(-\frac12 (x-\mu_{2})^T\Sigma_i^{-1}(x-\mu_2)\right)$$
Hence if the covariance are not identical for the classes, the boundary won't be linear. Assume that $\Sigma_1=\Sigma_2=\Sigma$,
$$\exp\left((\mu_1-\mu_2)^T\Sigma^{-1}x-\frac12 \mu_1^T\Sigma^{-1}\mu_1 +\frac12 \mu_2^T\Sigma^{-1}\mu_2\right)=\frac{f(C_2)}{f(C_1)}$$
Hence the equation of the line is : $$(\mu_1-\mu_2)^T\Sigma^{-1}x=\frac12 \mu_1^T\Sigma^{-1}\mu_1 -\frac12 \mu_2^T\Sigma^{-1}\mu_2+\ln(f(C_2))-\ln(f(C_1))$$