Despite a similar post here, I read that the entropy of a uniformly distributed discrete random variable is always log base $2$ of the number of observations in the dataset, $H(X) = \log(N)$. Is this also true for a uniformly distributed continuous r.v.?
2026-03-27 23:32:07.1774654327
Entropy of discrete and continuous uniform distributions
381 Views Asked by Bumbble Comm https://math.techqa.club/user/bumbble-comm/detail At
1
There are 1 best solutions below
Related Questions in PROBABILITY-DISTRIBUTIONS
- Given is $2$ dimensional random variable $(X,Y)$ with table. Determine the correlation between $X$ and $Y$
- Statistics based on empirical distribution
- Given $U,V \sim R(0,1)$. Determine covariance between $X = UV$ and $V$
- Comparing Exponentials of different rates
- Linear transform of jointly distributed exponential random variables, how to identify domain?
- Closed form of integration
- Given $X$ Poisson, and $f_{Y}(y\mid X = x)$, find $\mathbb{E}[X\mid Y]$
- weak limit similiar to central limit theorem
- Probability question: two doors, select the correct door to win money, find expected earning
- Calculating $\text{Pr}(X_1<X_2)$
Related Questions in RANDOM-VARIABLES
- Prove that central limit theorem Is applicable to a new sequence
- Random variables in integrals, how to analyze?
- Convergence in distribution of a discretized random variable and generated sigma-algebras
- Determine the repartition of $Y$
- What is the name of concepts that are used to compare two values?
- Convergence of sequences of RV
- $\lim_{n \rightarrow \infty} P(S_n \leq \frac{3n}{2}+\sqrt3n)$
- PDF of the sum of two random variables integrates to >1
- Another definition for the support of a random variable
- Uniform distribution on the [0,2]
Related Questions in INFORMATION-THEORY
- KL divergence between two multivariate Bernoulli distribution
- convexity of mutual information-like function
- Maximizing a mutual information w.r.t. (i.i.d.) variation of the channel.
- Probability of a block error of the (N, K) Hamming code used for a binary symmetric channel.
- Kac Lemma for Ergodic Stationary Process
- Encryption with $|K| = |P| = |C| = 1$ is perfectly secure?
- How to maximise the difference between entropy and expected length of an Huffman code?
- Number of codes with max codeword length over an alphabet
- Aggregating information and bayesian information
- Compactness of the Gaussian random variable distribution as a statistical manifold?
Related Questions in UNIFORM-DISTRIBUTION
- Uniform distribution: two parts of semicircle
- What is the distribution of the modular inverse of a uniformly random element in $\mathrm{Z}_{n}\setminus\{0\}$
- Determine limits for marginal pdf after Jacobian transformation
- distribution of Z=X+Y
- integrand of norm subjected to translation
- Convergence of ratio of two sums of uniform random variables
- Variance of $T_n = \min_i \{ X_i \} + \max_i \{ X_i \}$
- $X$ and $Y$ has uniform distribution. Find $(X-Y)^2$
- The sequence $\,a_n=\lfloor \mathrm{e}^n\rfloor$ contains infinitely many odd and infinitely many even terms
- Difference between conditional expectation E(Y|X) and E(Y|X=x)
Related Questions in ENTROPY
- Relation between Shanon entropy via relation of probabilities
- How to maximise the difference between entropy and expected length of an Huffman code?
- Appoximation of Multiplicity
- Two questions about limits (in an exercise about the axiomatic definition of entropy)
- Computing entropy from joint probability table
- Joint differential entropy of sum of random variables: $h(X,X+Y)=h(X,Y)$?
- What is the least prime which has 32 1-bits?
- Eggs, buildings and entropy
- Markov chains, entropy and mutual information
- Entropy and Maximum Mutual Information
Trending Questions
- Induction on the number of equations
- How to convince a math teacher of this simple and obvious fact?
- Find $E[XY|Y+Z=1 ]$
- Refuting the Anti-Cantor Cranks
- What are imaginary numbers?
- Determine the adjoint of $\tilde Q(x)$ for $\tilde Q(x)u:=(Qu)(x)$ where $Q:U→L^2(Ω,ℝ^d$ is a Hilbert-Schmidt operator and $U$ is a Hilbert space
- Why does this innovative method of subtraction from a third grader always work?
- How do we know that the number $1$ is not equal to the number $-1$?
- What are the Implications of having VΩ as a model for a theory?
- Defining a Galois Field based on primitive element versus polynomial?
- Can't find the relationship between two columns of numbers. Please Help
- Is computer science a branch of mathematics?
- Is there a bijection of $\mathbb{R}^n$ with itself such that the forward map is connected but the inverse is not?
- Identification of a quadrilateral as a trapezoid, rectangle, or square
- Generator of inertia group in function field extension
Popular # Hahtags
second-order-logic
numerical-methods
puzzle
logic
probability
number-theory
winding-number
real-analysis
integration
calculus
complex-analysis
sequences-and-series
proof-writing
set-theory
functions
homotopy-theory
elementary-number-theory
ordinary-differential-equations
circles
derivatives
game-theory
definite-integrals
elementary-set-theory
limits
multivariable-calculus
geometry
algebraic-number-theory
proof-verification
partial-derivative
algebra-precalculus
Popular Questions
- What is the integral of 1/x?
- How many squares actually ARE in this picture? Is this a trick question with no right answer?
- Is a matrix multiplied with its transpose something special?
- What is the difference between independent and mutually exclusive events?
- Visually stunning math concepts which are easy to explain
- taylor series of $\ln(1+x)$?
- How to tell if a set of vectors spans a space?
- Calculus question taking derivative to find horizontal tangent line
- How to determine if a function is one-to-one?
- Determine if vectors are linearly independent
- What does it mean to have a determinant equal to zero?
- Is this Batman equation for real?
- How to find perpendicular vector to another vector?
- How to find mean and median from histogram
- How many sides does a circle have?
Think about it.
A real number between $0$ and $1$ is able to contain infinite information. Just encode any symbol of the alphabet with the decimal representation 3 digit ascii symbols for example.
Entropy of continous distribution means how much information you need to store just for the shape of the distribution. You will need to add as many extra bits for whatever precision you will want.