Write a function of the log odds of the posterior probabilities (QDA) to see it is quadratic in x.

73 Views Asked by At

I'm reading the An Introduction to Statistical Learning book and get stuck at Chapter 4 page 159 where the author(s) state that in the QDA setting, the log odds of the posterior probabilities is quadratic in x:

$$ log \left( \dfrac{Pr(Y = k|X = x)}{Pr(Y = K|X = x)} \right) = a_k + \sum _{j=1}^{p} b_{kj} x_j + \sum _{j=1}^{p} \sum _{l=1}^{p} c_{kjl} x_j x_l$$

where $$ a_k, b_{kj}, c_{kjl} $$ are functions of $$ \pi_k , \pi_K, \mu_k, \mu_K, \Sigma_k, \Sigma_K $$

In exercise 11 (page 192), I am asked to work out the form of $$ a_k, b_{kj}, c_{kjl} $$ and I have no idea how to start. How can I solve this problem?

Thank you.

Link for the book: https://hastie.su.domains/ISLR2/ISLRv2_website.pdf

1

There are 1 best solutions below

0
On

Ratio of posteriors gives $$ \frac{\pi_k \sqrt{|\mathbf{\Sigma}_K|}} {\pi_K \sqrt{|\mathbf{\Sigma}_k|}} \exp \left[ - \frac12 (\mathbf{x}-\mathbf{m}_k)^T \mathbf{\Sigma}_k^{-1}(\mathbf{x}-\mathbf{m}_k) +\frac12 (\mathbf{x}-\mathbf{m}_K)^T \mathbf{\Sigma}_K^{-1}(\mathbf{x}-\mathbf{m}_K) \right] $$ Taking log yields $$ a + \frac12 \left[ (\mathbf{x}-\mathbf{m}_K)^T \mathbf{\Sigma}_K^{-1}(\mathbf{x}-\mathbf{m}_K) - (\mathbf{x}-\mathbf{m}_k)^T \mathbf{\Sigma}_k^{-1}(\mathbf{x}-\mathbf{m}_k) \right] = a + \frac12 \left[ \mathbf{x}^T \mathbf{Q} \mathbf{x} + \mathbf{b}^T \mathbf{x} + c \right] $$ You are asked to make explicit the scalars $a,c$, the vector $\mathbf{b}$ and the symmetric matrix $\mathbf{Q}$. I leave it to you to do the calculations from here.