What is the entropy of a doubly stochastic matrix?

199 Views Asked by At

On a doubly stochastic matrix, we can investigate the entropy of various components. For example, let us consider a list of items and a list of boxes; each item is in a box. Then there is a doubly stochastic matrix whose rows give the probability distribution that a particular item is in each different box, and whose columns give the probability distribution that a particular box contains each of the different possible items. We can talk about the entropy with respect to each item or location; is there an entropy for the entire matrix?

As an example, suppose that I have two boxes and two items, a red and blue ball. I flip a fair coin to determine which ball goes in which box. The doubly stochastic matrix would be:

$$ \begin{pmatrix} \frac{1}{2} & \frac{1}{2} \\ \frac{1}{2} & \frac{1}{2} \end{pmatrix} $$

I would hope to see $\ln{2}$ nats, or 1 bit of entropy, corresponding to the 1 bit from the fair flipped coin.