I read many times that averaging averages leads to errors, but here is my problem :
I'm trying to make a game simulation where a player is delving into a dungeon fighting monster—one monster per level. The player can resurrect two times, restarting the dungeon on the first floor.
I want to know, on average, how deep my player will get per game (so a group of 3 delves).
Let's imagine that I run the simulation for a total of 100 games. I could run the simulation and calculate a global average ( the total monsters killed / 3 / 100 ), but that seems wrong because I lose this "group of 3 delves per game" aspect.
Instinctively I want to make an average of averages: calculate the average level reached per game and average this average for the 100 games.
Am I wrong?
You are interested in the average level reached per game.
In each game, a player may resurrect $3$ times.
First, assume that no one ever finishes the game. That is, you assume that in each contest, a player uses all $3$ delves and then fails to complete the game. Then, since there is one monster killed per level, the average number of monsters killed equates to the intermediate level reached.
For example, assume:
There were $(100)$ games played.
In these $(100)$ games, a total of $(250)$ monsters were killed.
In each of the $(100)$ games, the player used all $(3)$ delves.
In each of the $(100)$ games, the player tried but failed to kill all the monsters. Here, I assume that the player died after using all $(3)$ delves in each contest.
Then, it would be reasonable to conclude that the average progress made during these $(100)$ games is $2.5$.
At this point, the analysis gets somewhat complicated. If I was attempting to quantify the expected depth that a player will reach, the first question that I would ask is:
How often does a player complete all of the game levels, without getting killed? For example, suppose that the game has $(10)$ levels, and in $(100)$ simulations, less than $(5)$ of the trials resulted in the player completing level $(10)$. Then, I would be content to accept the small analytical error involved in assigning a score of $(10)$ to these simulations. The error is that if the game had had more levels, in these handful of cases, the players would have killed more monsters.
If you find that (for example) more than $(20\%)$ of the players are completing the game, then you have to ask what the statistic you are trying to measure is supposed to signify. Does it indicate how far a player might have gotten if they had not run out of levels, or are you content to ignore that consideration?
As another consideration in considering the results, I would exclude any data that involved any situation where the player did not use all $(3)$ delves. The idea is that it is simply too difficult to accurately project how far a player would have gotten if they had decided to try to complete the game.
Excluding the considerations in the previous section, the computation of $$\frac{\text{total monsters killed}}{100}$$ seems like a reasonable description of the average depth a player will reach.
Note
Here, I am ignoring any consideration of whether the levels get more difficult as you go. That is, assuming that the levels do get more complex, I am assuming that you don't care. That is, I am assuming that you want to know how deep the typical player will get.
This means (for example), that you are willing to equate $(2)$ players each killing $(2)$ monsters and a third player-killing $(5)$ monsters as equivalent to all $(3)$ players killing $(3)$ monsters each.
If the levels do get more difficult, and if the assumption in the last paragraph does not meet your needs, then the analysis is necessarily more complicated. At this point, you would then have to ask what it is you are trying to measure.
For example, do you want to award (in effect) $(2)$ points for a monster killed at level $(5)$ versus only $(1)$ point for a monster killed at level $(2)$? Only you can answer this question, and only after asking yourself what you are trying to accomplish by considering how deeply the average player gets.