confusion about uniform probability distribution expectation value

Question

I read in my statistics book about Discrete Uniform Probability Distribution and the Expected Value

A discrete random variable $X$ with $k$ possible outcomes $x_1, x_2, ...,x_k$ is said to follow a discrete uniform distribution if the probability mass function (PMF) of $X$ is given by

$$P(X = x_i) = \frac{1}{k}, \forall i = 1,2,...,k $$

If the outcomes are the natural numbers $x_i = i (i = 1, 2, . . . , k)$, the mean and variance of $X$ are obtained as

$$E(X) = \frac{k+1}{2}, $$ $$Var(X) = \frac{1}{12}(k^2 - 1) $$

I have a question on $E(X)$. It seems I find it hard to understand the formula for $E(X)$ for coin toss example. in a fair coin toss we have $k = 2$ (heads or tails, here $0$ and $1$ respectivley) and $ E(X)= \frac{2+1}{2} = 1.5$

However, when you calculate the $E(X)$ as

$ E(X) = $ probability weighted sum of $k$ it is

$$ E(X) = 0\times0.5 + 1\times0.5 = 0.5 $$

So why is this difference? What am I missing?

Answer 1

The way you define the random variable $X$ is not consistent between the two approaches, which is why you have different results. In the discrete uniform case, the support is $X \in \{1, 2\}$, with probability mass function $$\Pr[X = 1] = 1/2, \\ \Pr[X = 2] = 1/2.$$ Hence the expectation is $$\operatorname{E}[X] = 1 \cdot \Pr[X = 1] + 2 \Pr[X = 2] = \frac{1}{2} + \frac{2}{2} = \frac{3}{2}.$$ When you try to represent the outcomes of a fair coin toss with $X$, you instead wrote $X \in \{0, 1\}$, hence $$\Pr[X = 0] = 1/2, \\ \Pr[X = 1] = 1/2.$$ Obviously, you will not get the same result.

Answer 2

The $k$ outcomes are numbered from 1 to $k$. In that case, the average value is indeed $(k+1)/2$.

However, if the $k$ outcomes are numbered from 0 to $k-1$, the average values is $k/2$. In that case, if you have two outcomes, 0 and 1, the average values is 0.5.

Note that this is the average value of number of heads that you get after throwing the coin once. Similarly, the average value of number of tails is the same.

Now, note that "tails" and "heads" are outcomes that are "labels" and not numbers. Therefore, it may make more sense to talk about the entropy of the outcomes rather than the variance.