partial derivative of cosine similarity

7.1k Views Asked by At

I asked a question about derivative of cosine similarity.

But no one has answered my question. Therefore I tried to do it my self as bellow.

$$ cossim(a,b)=\frac{a\cdot{b}}{\sqrt{a^2\cdot{b^2}}} \\\frac{cossim(a,b)}{\partial{a_1}}=\frac{\partial}{\partial{a_1}} \frac{a_1\cdot{b_1}+...+a_n\cdot{b_n}}{|a|\cdot|b|} \\=\frac{\partial}{\partial{a_1}}a_1\cdot{b_1}\cdot{(a_1^2+a_2^2+...a_n^2)^{-1/2}}\cdot{|b|^{-1}} \\= {b_1}\cdot{(a_1^2+a_2^2+...a_n^2)^{-1/2}}\cdot{|b|^{-1}}-a_1^2b_1(a_1^2+a_2^2+...a_n^2)^{-3/2} {|b|^{-1}} \\=\frac{b_1}{|a|\cdot{|b|}}-\frac{a_1|a|^{-2}\cdot{a_1b_1}}{|a|\cdot{|b|}} \\=\frac{b_1}{|a|\cdot{|b|}}-\frac{a_1\cdot{b_1}}{|a|\cdot{|b|}}\cdot{\frac{a_1} {|a|^2}} \\\therefore \frac{\partial}{\partial{a}}cossim(a,b)= \frac{b_1}{|a|\cdot{|b|}}-cossim(a,b)\cdot{\frac{a_1} {|a|^2}} $$

Is this correct?

1

There are 1 best solutions below

6
On BEST ANSWER

Putting $$ \cos (\mathbf{v},\mathbf{w}) = \frac{{\mathbf{v} \cdot \mathbf{w}}} {{\left| {\mathbf{v} } \right|\;\left| \mathbf{w} \right|}} $$ I would develop the required derivative as follows: $$ \cos (\mathbf{v} + d\mathbf{v},\mathbf{w}) = \frac{{\mathbf{v} \cdot \mathbf{w} + d\mathbf{v} \cdot \mathbf{w}}} {{\left| {\mathbf{v} + d\mathbf{v}} \right|\;\left| \mathbf{w} \right|}} $$ Now $\left| {\mathbf{v} + d\mathbf{v}} \right|$ can be rewritten as: $$ \begin{gathered} \left| {\mathbf{v} + d\mathbf{v}} \right| = \sqrt {\left( {\mathbf{v} + d\mathbf{v}} \right) \cdot \left( {\mathbf{v} + d\mathbf{v}} \right)} = \sqrt {\left| \mathbf{v} \right|^2 + \left| {d\mathbf{v}} \right|^2 + 2\mathbf{v} \cdot d\mathbf{v}} = \hfill \\ = \left| \mathbf{v} \right|\sqrt {1 + 2\frac{\mathbf{v}} {{\left| \mathbf{v} \right|^2 }} \cdot d\mathbf{v} + \frac{{\left| {d\mathbf{v}} \right|^2 }} {{\left| \mathbf{v} \right|^2 }}} \approx \left| \mathbf{v} \right|\left( {1 + \frac{\mathbf{v}} {{\left| \mathbf{v} \right|^2 }} \cdot d\mathbf{v}} \right) \hfill \\ \end{gathered} $$ hence: $$ \begin{gathered} \cos (\mathbf{v} + d\mathbf{v},\mathbf{w}) = \frac{{\mathbf{v} \cdot \mathbf{w} + d\mathbf{v} \cdot \mathbf{w}}} {{\left| {\mathbf{v} + d\mathbf{v}} \right|\;\left| \mathbf{w} \right|}} \approx \frac{{\mathbf{v} \cdot \mathbf{w} + d\mathbf{v} \cdot \mathbf{w}}} {{\left| \mathbf{v} \right|\left( {1 + \frac{\mathbf{v}} {{\left| \mathbf{v} \right|^2 }} \cdot d\mathbf{v}} \right)\;\left| \mathbf{w} \right|}} \approx \hfill \\ \approx \frac{{\mathbf{v} \cdot \mathbf{w} + \mathbf{w} \cdot d\mathbf{v}}} {{\left| \mathbf{v} \right|\;\left| \mathbf{w} \right|}}\left( {1 - \frac{\mathbf{v}} {{\left| \mathbf{v} \right|^2 }} \cdot d\mathbf{v}} \right) \approx \frac{{\mathbf{v} \cdot \mathbf{w}}} {{\left| \mathbf{v} \right|\;\left| \mathbf{w} \right|}} + \left( {\frac{\mathbf{w}} {{\left| \mathbf{v} \right|\;\left| \mathbf{w} \right|}} - \frac{{\mathbf{v} \cdot \mathbf{w}}} {{\left| \mathbf{v} \right|\;\left| \mathbf{w} \right|}}\frac{\mathbf{v}} {{\left| \mathbf{v} \right|^2 }}} \right) \cdot d\mathbf{v} = \hfill \\ = \cos (\mathbf{v},\mathbf{w}) + \left( {\frac{\mathbf{w}} {{\left| \mathbf{v} \right|\;\left| \mathbf{w} \right|}} - \cos (\mathbf{v},\mathbf{w})\frac{\mathbf{v}} {{\left| \mathbf{v} \right|^2 }}} \right) \cdot d\mathbf{v} \hfill \\ \end{gathered} $$ Therefore, apart from a typo, your derivation $$ \frac{\partial}{\partial{a_1}}cossim(a,b)= \frac{b_1}{|a|\cdot{|b|}}-cossim(a,b)\cdot{\frac{a_1} {|a|^2}} $$ looks to be correct.