Gaussian Discriminant Analysis

Question

A Gaussian discriminant analysis (GDA) is a type of generative learning algorithm. What kind of mathematics background do I need to understand what it is? Are there any books that explain this subject very well?

Answer 1

I suggest that you'd start with learning some basic concepts in probability theory: continuous and discrete random variables, probability density functions, mean, variance and estimating them from a sample. Learn how to fit a uni-variate Gaussian distribution to data.

Then you'll need to understand what is a multi-variate Gaussian distribution. For this, you'll need to understand what is a covariance matrix and how it defines the geometry of the Gaussian, which requires understanding the spectral decomposition of matrix.

In order to understand how the GDA criterion is derived, you'll need to understand some more linear algebra: matrix-vector multipication, matrix-matrix multiplication, matrix inverse, matrix transpose. You'll also need to understand some calculus: derivatives, using derivatives to finding the extremas of a function, deriving a function that has multiple arguments, and deriving a function that involves matrices (matrix calculus).