I'm currently working on Exercise 12.4 from "Elements of Statistical Learning." This exercise involves a K-class classification problem solved using Fisher's Discriminant Analysis (FDA). The goal is to demonstrate the following relationship:
$$||z - \bar{z}_k||^2 - ||z - \bar{z}_k'||^2 = ||x - \bar{x}_k||_W^2 - ||x - \bar{x}_k'||_W^2$$
Here, the symbol $\(||\cdot||_W\)$ represents the Mahalanobis distance associated with the covariance matrix $\bf{W}$. The variables are defined as follows:
- $z = \bf{U}^\top x$, where $\bf{U}$ consists of the first $\(K-1\)$ eigenvectors of $\bf{W}^{-1}\bf{B}$.
- $\bf{W}$ and $\bf{B}$ are the within-class and between-class covariance matrices, respectively.
- $\bar{x}_k$ represents the centroid of class $k$.
I understand that the key to solving this problem lies in expressing $\bf{W}$ in terms of vectors $\bf{U}$ and their orthogonal complements to ensure that the orthogonal components cancel out. However, I'm unsure about the specific steps to achieve this. Can someone please provide guidance or a step-by-step approach to prove this relationship? Your insights and assistance would be highly appreciated.