The "most likely" value of a random variable S with a binomial distribution with parameters $n$ and $p$ is $S = \langle np \rangle$?

1.3k Views Asked by At

I am working through Example 11.1.3 on p. 353 of the Second Edition of Thomas & Cover's "Information Theory":

We show that $\frac{1}{n+1} 2^{nH(k/n)} \le \binom{n}{k} \le 2^{nH(k/n)}$.

I am following the proof of the upper bound fine, but I do not understand the initial claim used to prove the lower bound. It says:

For the lower bound, let $S$ be a random variable with a binomial distribution with parameters n and p. The most likely value of $S$ is $S = \langle np \rangle$. This can easily be verified from the fact that $\frac{P(S = i+1)}{P(S = i)} = \frac{n-i}{i+1} \frac{p}{1-p}$ and considering the cases when $i < np$ and when $i > np$. Then, since there are $n + 1$ terms in the binomial sum, $1= \sum_{k=0}^n \binom{n}{k} p^k(1−p)^{n−k} \le (n+1) \max_k \binom{n}{k}p^k(1−p)^{n−k} = (n+1)\binom{n}{\langle np \rangle} p^{\langle np \rangle} (1-p)^{n - \langle np \rangle}$

Probability is not my strong suit and I am not sure what it means by the "most likely" value of $S$ is $S = \langle np \rangle$. I Googled "mode of binomial distribution" and found that the mode is $\lfloor (n+1)p\rfloor$ or $\lceil(n+1)p\rceil - 1$. Is $\langle np \rangle$ common notation for this or am I misunderstanding what is being said? This notation is not explained in the "list of symbols" in the back of the textbook either.

Any insight would be appreciated.

1

There are 1 best solutions below

0
On BEST ANSWER

Let us denote the most likely value of $S$ as $i_m$.

If $i_m$ is the most likely point then the two conditions apply

\begin{align} P(S = i_m) \geq P(S = i_m - 1) \\ P(S = i_m) \geq P(S = i_m + 1) \end{align}

which implies

\begin{align} \frac{P(S = i_m)}{P(S = i_m - 1)} &\geq 1\\ \frac{n - (i_m - 1)}{(i_m - 1) + 1} \cdot \frac{p}{1 - p} &\geq 1 \\ \implies i_m \leq np + p \end{align}

and also

\begin{align} \frac{P(S = i_m + 1)}{P(S = i_m)} &\leq 1 \\ \frac{n - i_m}{i_m + 1} \cdot \frac{p}{1 - p} &\leq 1 \\ \implies np - (1 - p) &\leq i_m \end{align}

combining the two results gives $ np - (1 - p) \leq i_m \leq np + p$.

$(1 - p)$ and $p$ are both numbers in $(0,1)$ so as $n$ tends to $\infty$, $i_m$ tends to $np$.