Entropy of cosine similarity etc.

Question

First, I would like to know how to calculate entropy of cosine similarity.

We have $H(X)=-\sum P(x_i)log_bP(x_i)$ for entrpy, and

similarity $ = \sum \frac{A_iB_i}{\sqrt{\sum(A_i^2)}\sqrt{\sum(B_i^2)}} $,

but we have two variable A,B for cosine distance, whereas we have only x for entropy. How could I calculate entropy?

Answer 1

Let $A,B$ be random vectors of $n$ componentes.

Let the similarity be denoted by $S = \sum \frac{A_iB_i}{\sqrt{\sum(A_i^2)}\sqrt{\sum(B_i^2)}}$. This variables has a discrete support assuming than $A$ and $B$ are discrete.

Now we want to compute the entropy of $S$; that is,

$$ H(S) = -\sum P(x_i)log_bP(x_i) $$

How can we compute it? Well, we cannot. Not without knowing the distributions of $A$ and $B$ at least.