Consider the following problem: Let $X,X_1,X_2,...$ be i.i.d. random variables. We execute the following experiment. One samples $X$. Then, one samples $X_1$,$X_2$ and so on until the first time the initial value $X$ was surpassed (or at least repeated).
Denote by $T$ the number of tries needed until this experiment is terminated. I wish to find out what the expectancy of $T$ is.
It is clear that, conditioned on ${X=x}$, $T$ is a geometric random variable with expectation $\frac{1}{\Pr\{X\ge x\}}$.Thus, if $X$ was absolutely continuous with density $f_X$, its expectation could be written as an integral: $$ \int_{-\infty}^{\infty} {\frac{f_X(x)}{1-F_X(x)}dx} $$ where $F_X(x)=\Pr\{X\le x\}$. Noting that $\frac{d}{dx}F_X=f_X$ yields a formula for a primitive function which is -$\log(1-F_X(x))$, which diverges as $x\rightarrow \infty$, so the expecation is infinite.
Similarly, if $X$ was discrete then we could assume WLOG that it is supported on the integers. Letting $\Pr\{X=n\}=p_n$, we have the following formula for the expectation: $$ \mathbb{E}T=\sum_{n=-\infty}^{\infty} {\frac{p_n}{\sum_{k=n}^{\infty} p_k}} $$
Questions:
Suppose $X$ is supported on a finite set. It is clear that $T$ has finite expectation in this case. Can one provide a nicer form of the expression above?
It's easy to see that the limit $n\rightarrow -\infty$ "behaves nicely" in the sum above (as the denominator goes to $1$ and the numerator converges). If one assumes that $p_n$ does not vanish for large enough values of $n$, is it true that the series diverges?
What happens if $X$ is continuous but not absolutely continuous? Is there a unified theory all random variables?
Thanks.
@Did answered (2) for you.
Ill try (1):
Let $X(\omega) \in I,\;|I|=M<\infty,\;\text{ and } P(X=i)=p_i\; \forall i\in I$:
Lets say $X=j: j\in I$ then:
$$E[T|X=j]=\frac{1}{\sum\limits_{i\in I:i\geq j}p_j}$$
By the tower property of conditional expectation:
$$E[E[T|X]]=E[T]$$ so
$$E[T]=\sum_{i\in I} \left\{P(X=i)E[T|X=i]\right\}=\sum_{i\in I}\frac{p_i}{\sum\limits_{j\in I:j\geq i}p_j}\leq \sum_{i\in I} \left\{p_i\cdot \frac{1}{p_i}\right\}=|I|=M<\infty$$
$\square$
For (3): Yes, it is called measure theory.