Question

Confusion regarding the non-negativity of the relative entropy and how can $H(X|Y) \le H(X)$

score 172 · Answer 1 · 2020-08-02 13:12:32

172

Views

Confusion regarding the non-negativity of the relative entropy and how can $H(X|Y) \le H(X)$

Published on 02 Aug 2020 - 13:12

score 73 · Answer 2 · 2020-08-05 11:19:25

73

Views

Matrix notation for entropy

Published on 05 Aug 2020 - 11:19

score 338 · Answer 3 · 2020-08-05 21:10:53

338

Views

Name of a quantity related to Shannon entropy, but with squared logarithm

Published on 05 Aug 2020 - 21:10

score 46 · Answer 4 · 2020-08-07 13:00:15

46

Views

Calculating the number of ways a set of $n$ numbers with $k$ different possible values can yield a specific average of $m$.

Published on 07 Aug 2020 - 13:00

score 866 · Answer 5 · 2020-08-08 13:01:43

866

Views

Difference between joint probability and mutual information on a Venn Diagram

Published on 08 Aug 2020 - 13:01

score 49 · Answer 6 · 2026-03-26 18:59:49

49

Views

Proving $S(A:B)+S(A:C) \le 2S(A)$ using subadditivity

Published on 26 Mar 2026 - 18:59

score 316 · Answer 7 · 2020-08-11 17:58:02

316

Views

Interpreting Fano's Inequality

Published on 11 Aug 2020 - 17:58

score 466 · Answer 8 · 2026-02-23 11:49:07

466

Views

What do the level sets of the Shannon entropy look like?

Published on 23 Feb 2026 - 11:49

score 90 · Answer 9 · 2020-08-16 09:41:13

90

Views

Does entropy increase exponentially as the number of observations increase?

Published on 16 Aug 2020 - 9:41

score 616 · Answer 10 · 2026-03-26 12:35:39

616

Views

Subadditivity property of relative entropy

Published on 26 Mar 2026 - 12:35

score 122 · Answer 11 · 2020-08-20 12:41:37

122

Views

What is constraining the Kullback-Leibler divergence to not be infinitely negative?

Published on 20 Aug 2020 - 12:41

score 46 · Answer 12 · 2020-08-20 20:34:55

46

Views

Could you please help me in verifying if the calculation of entropy in the given question is correct?

Published on 20 Aug 2020 - 20:34

score 127 · Answer 13 · 2026-02-23 06:00:36

127

Views

Identical CDF over sets of measure zero.

Published on 23 Feb 2026 - 6:00

score 1k · Answer 14 · 2020-08-25 08:07:06

1k

Views

derivative of the $L^p$-norm with respect to $p$

Published on 25 Aug 2020 - 8:07

score 456 · Answer 15 · 2020-08-28 13:11:21

456

Views

What's the maximum entropy for a discrete distribution with non negative support and a given mean and variance?

Published on 28 Aug 2020 - 13:11

List Question

Confusion regarding the non-negativity of the relative entropy and how can $H(X|Y) \le H(X)$

Matrix notation for entropy

Name of a quantity related to Shannon entropy, but with squared logarithm

Calculating the number of ways a set of $n$ numbers with $k$ different possible values can yield a specific average of $m$.

Difference between joint probability and mutual information on a Venn Diagram

Proving $S(A:B)+S(A:C) \le 2S(A)$ using subadditivity

Interpreting Fano's Inequality

What do the level sets of the Shannon entropy look like?

Does entropy increase exponentially as the number of observations increase?

Subadditivity property of relative entropy

What is constraining the Kullback-Leibler divergence to not be infinitely negative?

Could you please help me in verifying if the calculation of entropy in the given question is correct?

Identical CDF over sets of measure zero.

derivative of the $L^p$-norm with respect to $p$

What's the maximum entropy for a discrete distribution with non negative support and a given mean and variance?

Trending Questions

Popular # Hahtags

Popular Questions