Wikipedia’s solution of extending Fisher LDA

84 Views Asked by At

The Wikipedia article on the Kernel Fisher Discriminant Analysis (https://en.wikipedia.org/wiki/Kernel_Fisher_discriminant_analysis) states under the heading “Extending LDA” that $\alpha=N^{-1}\cdot(M_{2}-M_{1})$ is a solution of the following equation:

\begin{equation} [\alpha^T \cdot \underbrace{(M_{2}-M_{1}) \cdot (M_{2}-M_{1})^T}_{\substack{=M}} \cdot \alpha] \cdot N \cdot \alpha = [\alpha^T \cdot N \cdot \alpha] \cdot \underbrace{(M_{2}-M_{1}) \cdot (M_{2}-M_{1})^T}_{\substack{=M}} \cdot \alpha. \end{equation}

Inserting the proposed solution into this equation yields:

\begin{equation} [(M_{2}-M_{1})^T \cdot (N^{-1})^T \cdot (M_{2}-M_{1}) \cdot (M_{2}-M_{1})^T \cdot N^{-1}\cdot(M_{2}-M_{1})] \cdot \color{magenta}{N \cdot N^{-1}} \cdot(M_{2}-M_{1}) = [(M_{2}-M_{1})^T \cdot (N^{-1})^T \cdot \color{magenta}{N \cdot N^{-1}} \cdot(M_{2}-M_{1})] \cdot (M_{2}-M_{1}) \cdot (M_{2}-M_{1})^T \cdot N^{-1}\cdot(M_{2}-M_{1}). \end{equation}

Simplifying further (ie, $\color{magenta}{N \cdot N^{-1} = E}$) results in the following:

\begin{equation} [\color{green}{(M_{2}-M_{1})^T \cdot (N^{-1})^T \cdot (M_{2}-M_{1})} \cdot \color{blue}{(M_{2}-M_{1})^T \cdot N^{-1}\cdot(M_{2}-M_{1})}] \cdot \color{red}{(M_{2}-M_{1})} = [\color{green}{(M_{2}-M_{1})^T \cdot (N^{-1})^T \cdot (M_{2}-M_{1})}] \cdot \color{red}{(M_{2}-M_{1})} \cdot \color{blue}{(M_{2}-M_{1})^T \cdot N^{-1}\cdot(M_{2}-M_{1})}. \end{equation}

The $\color{green}{green}$ product transposed is equal to the $\color{blue}{blue}$ product. As matrix multiplication is generally not commutative, I have a hard time understanding this equality. More precisely, I do not understand why the $\color{red}{red}$ term can be simply moved.

1

There are 1 best solutions below

1
On BEST ANSWER

$(M_2 - M_1)^T \cdot N^{-1} \cdot (M_2 - M_1)$ is $1 \times 1$ matrix, so pretty much a scalar, which is why it can commute with everything, including $(M_2 - M_1)$.