This blog, Shannon entropy, by Yurii Lahodiuk shows the link (derivation) of Shannon entropy from basic combinatorics. I would like to know the first person that made this combinatorial interpretation of Shannon entropy.
Who is the first person who gave this interpretation? What is the first appearance of this combinatorial interpretation in the literature?
On
According to Wikipeida, this interpretation (known as the maximum entropy principle) was given by Jaynes (based on a suggestion by Wallis). Here is an excerpt:
The principle was first expounded by E. T. Jaynes in two papers in 1957 where he emphasized a natural correspondence between statistical mechanics and information theory. In particular, Jaynes offered a new and very general rationale why the Gibbsian method of statistical mechanics works. He argued that the entropy of statistical mechanics and the information entropy of information theory are basically the same thing.
I have written the blog post: http://lagodiuk.github.io/computer_science/2016/10/31/entropy.html.
Frankly speaking, I have learnt about this interpretation from the exam-preparation notes, which I have found here: https://github.com/dmitriykovalev/nsu.videosoft.org/blob/master/assets/content/pdf/entropy.pdf
Which seems to be a part of the course taught by Kovalev Dmitry Sergeyevich "Data presentation and compression": http://nsu.videosoft.org/2009/.