Mean of the Geometric random variable

Question

I am following the book "Introduction to probability" from Bertsekas.

In the book the derivation of the mean of Geometric random variable is through the use of the Total expecation theorem which involves conditional expectation.

My problem is when I try to derive $E[X]$ I end up getting $E[X] = E[X]$ instead of $E[X] = \frac{1}{p}$

I am going to try to derive the mean. Please highlight where I may be wrong. I am still new to probability so please highlight any small mistakes as well.

$E[X] = \sum_{k=1}^\infty ( P(k) \times E[X | k]) = P(k = 1) \times E[X | k = 1] + P(k > 1) \times E[X | k > 1]$

$P(k = 1) = p$

$P(k > 1) = 1 - p$ using sum of infinite geometric series formula

$E[X | k = 1] = 1 \times P(X | k = 1) = \frac{P(X \cap k = 1)}{P(k = 1)} = \frac{p}{p} = 1 $

The trouble is when I try to find $E[X | k > 1]$

$E[X | k > 1] = \sum_{k=2}^\infty ( k \times (P[X | k > 1]) $

$E[X | k > 1] = \sum_{k=2}^\infty ( k \times \frac{P(X \cap k > 1)}{P(k > 1)})$

$E[X | k > 1] = \sum_{k=2}^\infty ( k \times \frac{P(X \cap k > 1)}{(1-p)})$

$P(X \cap k > 1) = \sum_{k=2}^\infty ((1-p)^{k-1} \times p)$

I suspect the problem to be in the following line

$E[X | k > 1] = \frac{1}{(1-p)}\sum_{k=2}^\infty ( k \times \sum_{k=2}^\infty ((1-p)^{k-1} \times p)$

$E[X] = \sum_{k=1}^\infty ( k \times (1-p)^{k-1} \times p $

$E[X] = p + \sum_{k=2}^\infty ( k \times (1-p)^{k-1} \times p $

$\sum_{k=2}^\infty ( k \times (1-p)^{k-1} \times p = E[X] - p $

$E[X | k > 1] = \frac{E[X] - p}{1 - p}$

finally using the total expectation theorem

$E[X] = P(k = 1) \times E[X | k = 1] + P(k > 1) \times E[X | k > 1]$

$E[X] = p \times 1 + (1 - p) \times \frac{E[X] - p}{1 - p}$

$E[X] = E[X]$ ?? what is the meaning of this?

Thanks.

Answer 1

Let $S$ denote the event that the first attempt is successful. Then we can write:$$\mathbb EX=P(S)\mathbb E[X\mid S]+P\left(S^{\complement}\right)\mathbb E\left[X\mid S^{\complement}\right]=p\mathbb E[X\mid S]+(1-p)\mathbb E\left[X\mid S^{\complement}\right]\tag1$$

Now realize that $\mathbb E[X\mid S]=1$ (i.e. under the condition of a successful first attempt the expectation of the number of attempts needed equals $1$).

Further realize that $\mathbb E\left[X\mid S^{\complement}\right]=1+\mathbb EX$ (under condition of a first failing attempt we have one failure in our pocket and just start over again).

Substituting in $(1)$ we get:$$\mathbb EX=p+(1-p)(1+\mathbb EX)$$

This is an equality in $\mathbb EX$ that can easily be solved, leading to:$$\mathbb EX=\frac1p$$

With this method we find the expectation on an elegant way and only using the "character" of geometric distribution.

---

Observe that the first equality of $(1)$ can also be written as:$$\mathbb EX=P(X=1)\mathbb E[X\mid X=1]+P(X>1)\mathbb E\left[X\mid X>1\right]$$

which has resemblance with the first lines in your effort.

Your notation is IMV confusing.

Is $k$ an index (as notation $\sum_{k=1}^{\infty}\dots$ suggests) or is it a random variable (as notation $P(k=1)$ suggests)?...

---

edit (meant as confirmation of $\mathbb E[X\mid X>1]=1+\mathbb EX$)

$$\begin{aligned}\mathbb{E}\left[X\mid X>1\right] & =\sum_{k=2}^{\infty}kP\left(X=k\mid X>1\right)\\ & =\sum_{k=2}^{\infty}k\frac{P\left(X=k\text{ and }X>1\right)}{P\left(X>1\right)}\\ & =\sum_{k=2}^{\infty}k\frac{P\left(X=k\right)}{P\left(X>1\right)}\\ & =\sum_{k=2}^{\infty}k\frac{P\left(X=k\right)}{1-P\left(X=1\right)}\\ & =\sum_{k=2}^{\infty}k\frac{P\left(X=k\right)}{1-p}\\ & =\sum_{k=2}^{\infty}k\left(1-p\right)^{k-2}p\\ & =\sum_{k=1}^{\infty}\left(1+k\right)\left(1-p\right)^{k-1}p\\ & =\sum_{k=1}^{\infty}\left(1-p\right)^{k-1}p+\sum_{k=1}^{\infty}k\left(1-p\right)^{k-1}p\\ & =1+\mathbb{E}X \end{aligned} $$

Answer 2

Before looking through your long solution, may I suggest you a simply way.

Remember that, for non negative rv, the mean can also be defined as follows

• continuous variable

$$\mathbb{E}[X]=\int_0^{\infty}[1-F_X(x)]dx$$

• in your case (discrete) this become

$$\mathbb{E}[X]=\mathbb{P}[X>x]=\sum_{x=1}^{\infty}(1-p)^{x-1}=\frac{1}{1-(1-p)}=\frac{1}{p}$$

Answer 3

tommik's answer is the most convenient, but if you have not encountered the derivation of an expected value from the so-called survival function ($S(x) = 1-F(x)$) then you can still find the expected value from the more common definition:

$$\mathsf{E}[X] = \sum_{k=1}^\infty kP(X=k) = \sum_{k=1}^\infty k(1-p)^{k-1}p = p\sum_{k=1}^\infty k(1-p)^{k-1} \tag{1}$$

Note here that $$\sum_{k=1}^\infty k(1-p)^{k-1} = -\cfrac{\mathrm d}{\mathrm dp} \sum_{k=1}^\infty(1-p)^k = -\cfrac{\mathrm d}{\mathrm dp} \cfrac{1-p}{1-(1-p)} = -\cfrac{\mathrm d}{\mathrm dp} \left(\cfrac 1p-1\right) = \cfrac{1}{p^2}$$

$$\therefore (1) = \cfrac p{p^2} = \cfrac 1p$$

Answer 4

Firstly, since $\mathsf P(X=k)=(1-p)^{k-1}p\mathbf 1_{k\in\Bbb N^+}$, therefore $\mathsf P(X=1)= p, \mathsf P(X>1)=1-p$.

Now $\mathsf P(X\mid X=1)=1$ because, $1$ is the expected value of $X$ when $X=1$ .

That leaves $\mathsf E(X\mid X>1)$ which is $1$ failure plus the expected value of the count of trials after the first trial and until the first success of a series of Bernoulli trials with success rate $p$ ~ that later term is a random variable with the same distribution as $X$. So $\mathsf E(X\mid X>1)=1+\mathsf E(X)$

You have $$\begin{align}\mathsf E(X)&=\mathsf P(X=1)~\mathsf E(X\mid X=1)+\mathsf P(X>1)~\mathsf E(X\mid X>1)\\[1ex]&=p+(1-p)(1+\mathsf E(X))\\[2ex](1-(1-p))\mathsf E(X)&=p+1-p\\[3ex]\therefore~~\mathsf E(X)&=1/p\end{align}$$