Probability and Variance--Bernoulli, Binomial, Negative Binomial

Question

I am taking a course in probability and I have trouble computing the variance of a random variable.

There are 2 cases we saw in class that I would like to understand:

First I will state the definition of variance of expected value:

• If $X$ is a random variable who's values are in $\mathbb{N}$, then $$\mathbb{E}(X) = \Sigma_{n≥0}\mathbb{P} (X > n)$$

• If $X$ is a random variable and $\mathbb{E}(X)$ exists, then the variance of X is: $$Var(X) = \mathbb{E}\Big((X-\mathbb{E}(X))^2\Big)$$

Now here are the examples I'd like to understand:

1. binomial distribution: If $X$ is a random variable that follows a binomial distribution of parametres $n$ and $p$ then we can write $X = X_1+X_2+X_3+...+X_n$ where the $X_i$'s are bernoulli variables of parametre $p$. Then $$Var(X) = np(1-p)$$

2. negative binomial distribution (Pascal law): If X is a random variable that follows a pascal law of parameters $r$ and $p$ the $X+r= X_1+...+X_r$ where $X_i$'s are independant geometric variables. Then $$Var(X)=Var(X+r) = r\dfrac{1-p}{p^2}$$

I don't understand how one comes up with such results using the defintion of variance.

Answer 1

If $X$ is Bernoulli, with $P(\text{Success}) = p,$ then $E(X) = (0)(1-p) + 1p = p.$ Because $0^2 = 0$ and $1^2 = 1$, we also have $E(X^2) = p.$ From @Dave's hint $Var(X) = p - p^2 = p(1-p).$

Binomial: Then for $Y \sim Binom(n, p),$ we have $$E(Y) = E\left(\sum_{i=1}^n X_i \right) = \sum_{i=1}^n E(X_i) = np.$$

Also, direct proofs, with $$E(Y) = \sum_{i=0}^n i{n \choose i}p^i(1-p)^{n-i} = \sum_{i=1}^n i{n \choose i}p^i(1-p)^{n-i} = \cdots = np(1) = np,$$ (with a change in index $j = i-1$) are given in many elementary texts.

Similarly, using independence, $$Var(Y) = Var\left(\sum_{i=1}^n X_i \right) = \sum_{i=1}^n Var(X_i) = np(1-p).$$

Again here, proofs using about the same method as above to find $E(Y(Y-1)),$ and then $Var(Y)$ from that and $E(Y),$ are given in many elementary texts.

For the negative binomial, it seems intuitive that the avarage waiting time for the $r$th Success should be $r/p.$ A rigorous derivation of the expectation of a negative binomial random variable often uses some sort of trick involving differentiation of a sum. Probably the easiest route is to find the moment generating function and differentiate it to get the mean and variance.

Note: You may find some useful details on Wikipedia 'negative binomial distribution'. But if you are looking at various references on negative binomial, be aware that some parameterizations count trials until the $r$th Success and others count only the Failures before the $r$th Success.