When I read my textbook, I find one conclusion which states that if the mean of geometric random variable is finite, then with probability 1, we can get the first success within "finite" step.
I am confused about the "finite" part. Why there is no chance that we will never get success? Since the mean of geometric random variable is 1/p, doesn't it mean we can always get success within finite experiments if p > 0?
Probabilities are weights assigned to events, subsets of outcomes. Some events have zero weight.
For example, let's consider the event $A$ = "no success" = "no success in any finite number of steps". This event is contained in the event $A_n$ = "no success in the first $n$ experiments". The probability of the latter is $(1-p)^n$. Therefore by monotonicy of probabilities, $P(A) \le P(A_n)=(1-p)^n$ for every $n$. If $p>0$, this implies $P(A)=0$, that is, the weight assigned to "no success" is zero, or, equivalently, the weight assigned to "success in (some) finite time" is $1$.