Assume that I roll the dice until the average of the outcomes is $2.5$. I want to calculate the probability of stopping rolling the dice.
Let $E$ be the event of stop rolling the dice and $E_n$ be the event of getting the average $2.5$ after rolling the dice exactly $n$ times. Then, $E = E_1 \cup E_2 \cup E_3 \cup E_4 \cup \cdots$. Since it will not end with odd trials, I can rewrite it as $E = E_2 \cup E_4 \cup \cdots$.
Since ${E_{2n}}^\complement \supseteq \bigcup_{k=2n}^{\infty} E_{2k}$, $E_{2k}$'s are disjoint. Thus, $Pr(E) = \sum_{k=1}^{\infty} Pr(E_{2k})$. Thus, I can infer that $Pr(E) > Pr(E_2) = 1/9 >0$. Also, it seems that $\lim\limits_{n \to \infty } Pr(E_n) = 0 $ (I am also having trouble proving this), so my conjecture is $Pr(E)<1$.
However, I am stuck with calculating the exact probability of each event $E_{2k}$. How can I calculate the probability of each event $E_{2k}$ ($Pr(E_{2k})$ for $k>1$) and finally the probability of stopping rolling the dice $Pr(E)$? Also, I wonder if there is any related theorem that I can use to calculate such events.
Let $S_{2k}$ be the sum of $2k$ dice rolls. What you need then is the number of solutions to $\frac{S_{2k}}{2k}=2.5$, or $S_{2k}=5k$. This is easily found with generating functions, as illustrated below:
Let $g(x)=(x^1+x^2+x^3+...+x^6)^{2k}$ be our generating function. Each factor represents all the outcomes possible from rolling one die; the face values are represented by the various powers of $x$, from $1$ to $6$. What we want is $[x^{5k}]g(x)$; that is, the coefficient of $x^{5k}$ in the above expansion.
This problem is best tackled with a few simplifying steps. First we factor out an $x$ from each factor; with $2k$ factors in total we end up with $g(x)=x^{2k}(1+x+x^2+...+x^5)^{2k}$. We then rewrite the factor on the right using the identity $\sum_{i=0}^{n}x^i = \frac{1-x^{n+1}}{1-x}$. Doing this, we get $g(x)=x^{2k}(\frac{1-x^6}{1-x})^{2k}=x^{2k}(1-x^6)^{2k}(1-x)^{-2k}.$
The final simplifying step is to realize that factoring out $x^{2k}$ is equivalent to shifting all exponents by $2k$ to the left; in other words, finding $[x^{5k}]g(x)$ is equivalent to finding $[x^{3k}](1-x^6)^{2k}(1-x)^{-2k}$
That hardly seems simpler, does it? Not to worry — finding the coefficients is a simple task, while I will illustrate with a few examples. Suppose $k=1$; then we want $[x^{3}]=(1-x^6)^2(1-x)^{-2}$. We can think of the solution to this as "picking $x$'s from each factor such that the sum of all the exponents is, in the end, $3$, with the exponent on each factor representing the number of choices available. For example, we cannot pick any $x$ terms from $(1-x^6)^2$, because $6\gt 3$. They must come from the factor of $(1-x)^{-2}$. The number of ways of doing this is given by $\binom{-2}{3}(-x)^3 = \binom{2+3-1}{3}(-1)^3(-x)^3=\binom{4}{3}=4$. In other words, there are $4$ ways to roll $2$ dice such that their average is $2.5$: $(1,4),(4,1),(2,3)$ and $(3,2)$. With $6^2=36$ total outcomes possible (not all distinct, but that doesn't matter), the probability of the game ending with just two rolls is $\frac{4}{36}=\frac{1}{9}$. Notice also that $(-x)^3$ seemed to "disappear" during the calculation. That's because we don't care about $x^3$ itself, just its coefficient (which is $-1$).
For $k=2$, we want to find $[x^6]=(1-x^6)^4(1-x)^{-4}$. We could pick one lot of $(-x^6)$ from the first factor and no $x$ from the second; this would give $\binom{4}{1}(-1)$. Alternatively, we could pick no $x$ from the first factor and six lots of $(-x)$ from the second; this gives $\binom{-4}{6}(-1)^6=\binom{9}{6}=84$. Overall, we have $84-4=80$ ways of rolling $4$ dice such that the average is $2.5$ (which I am obviously not going to enumerate, lol). The probability of the game ending in $4$ rolls is therefore $\frac{80}{6^4} = \frac{5}{81}$
In general, for each value of $k$, $[x^{3k}]\frac{(1-x^6)^{2k}(1-x)^{-2k}}{6^{2k}}$ gives the probability of the game ending in exactly $2k$ throws of a die, i.e. of achieving an average of exactly $2.5$ in exactly $2k$ throws. Since you ultimately want the union of these events, you would apply summation to this expression, with $k$ ranging from $0$ to $\infty$.