In fisher linear descriminant, how to maximize J(w) when taking its derivative with respective to w

119 Views Asked by At

I am reading the book "Bishop - Pattern Recognition And Machine Learning" and I am confused about how to maximize J(w) in fisher linear discriminant. Here, $$J(w)=\frac{w^TS_Bw}{w^TS_Ww}$$ All the elements in this formula is vectors. when we take its derivative with respect to w and equate to zero, we get: $$(w^TS_Bw)S_Ww=(w^TS_Ww)S_Bw\tag{*}\label{*} $$ I cannot figure out how to do this. Here is my work: \begin{align} \frac{dJ(w)}{dw}&=\frac{d}{dw}\frac{w^TS_Bw}{w^TS_Ww}\\ &=\frac{(w^TS_Bw)'w^TS_Ww-w^TS_Bw(w^TS_Ww)'}{(w^TS_Ww)^2} \end{align} So, we just need the following equation because the derivative needs to equal 0. $$(w^TS_Bw)'w^TS_Ww=w^TS_Bw(w^TS_Ww)'$$, which is $$(S_B+S_B^T)ww^TS_Ww=w^TS_Bw(S_W+S_W^T)w$$$$S_Bww^TS_Ww+S_B^Tww^TS_Ww=w^TS_BwS_Ww+w^TS_BwS_W^Tw\tag{1}\label{1} $$ Then I am stuck. I do not know how to get the formula * from (1). Could anyone please help me out here? Because maths is a big issue for me, please provide as many details as possible. Thanks a lot.

1

There are 1 best solutions below

0
On BEST ANSWER

I figure it out. Here, $$S_B=S_B^T$$ and $$S_W=S_W^T$$ So the formula above (1) becomes: $$(2S_B)ww^TS_Ww=w^TS_Bw(2S_W)w $$ The rest is easy to understand.