How do you figure the average value of a variable affected by a repeated probable event ?.
Suppose I have an experiment with input costs of $100$ for each trial:
- The first trial has a $90\ \%$ chance of success,
- the second $80\ \%$ and
- the third $70\ \%$.
If I want to succeed at all three in a row, what cost should I expect, on average ?.
- I thought it was $300$, plus ( probability of failure * cost ).
- The probability of getting through three in a row is $0.9 * 0.8 * 0.7= 50.4$.
- So I figured the average cost at the $300$ plus $300/50.4$. That's $595.24$.
But when I run the simulation, I get an average cost of $520$. I'm missing something.
How do I correct the formula for figuring the expected cost of succeeding at three trials, with $90\ \%$, $80\ \%$ and $70\ \%$ probabilities of success and a cost of $100$ for each trial ?.
Let $X$ be the random number of trials needed to obtain three consecutive successes. Note the following assumption: if there is a failure before the third consecutive successful trial, then we start all over, in which the next trial's probability of success (immediately after the observed failure) is $0.9$. We wish to compute $\operatorname{E}[X]$, the mean number of trials needed. To this end, $$\operatorname{E}[X] = (0.9)(0.8)(0.7)(3) + (0.1)(1 + \operatorname{E}[X]) + (0.9)(0.2)(2 + \operatorname{E}[X]) + (0.9)(0.8)(0.3)(3 + \operatorname{E}[X]),$$ from which it follows that $$\operatorname{E}[X] = \frac{655}{126} \approx 5.19841.$$ At a cost of $100$ per trial this translates to an average cost of $519.841$ which agrees with your simulation.
Why does this formula work, and what does it mean? Well, look at it this way. If the first three trials are successful with probability $(0.9)(0.8)(0.7)$, then it took $3$ trials. Otherwise, the first trial failed with probability $0.1$, in which case we start over and the expected number of additional trials remains the same, so the total expected number of trials after the failure of the first trial is $1+\operatorname{E}[X]$. This explains the first two terms on the right-hand side.
If the first trial is successful but the second fails (and this has a probability of $(0.9)(0.2)$ of occurring), then again we start all over and the expected number of additional trials is again $\operatorname{E}[X]$, but this time we used $2$ trials, so the total expected number of trials after the failure of the first two is $2+\operatorname{E}[X]$.
Finally, if the first two trials are successful but the third fails, then we spent three trials but need to start over.
Notice that $$(0.9)(0.8)(0.7) + (0.1) + (0.9)(0.2) + (0.9)(0.8)(0.3) = 1$$ so we counted all the possible cases.
The model we are actually working with is a discrete-time Markov chain model with transition matrix $$\mathcal P = \begin{bmatrix}0.1 & 0.9 & 0 & 0 \\ 0.2 & 0 & 0.8 & 0 \\ 0.3 & 0 & 0 & 0.7 \\ 0 & 0 & 0 & 1 \end{bmatrix},$$ where each row corresponds to a state of the system in which there are a certain number of consecutive successes. So the first row corresponds to the state where we have no successes; the second row is for when we have had one success; and so forth, until the final row is the state where we have achieved three consecutive successes. The columns represent the state we are transitioning to, from a given row, and the value in a cell represents the probability of transitioning from one state to another. So for example, in the second row, third column, the value $0.8$ represents the conditional probability that, given we have observed one success so far (row 2), we observe another success in the next trial, thus transitioning to row 3. The value $0.3$ in the third row, first column, represents the conditional probability that, given we have observed two consecutive successes, we fail on the third trial and therefore must start over with zero consecutive successes (row 1). The last row and column has a $1$, because it represents the state in which we have obtained three consecutive successes, and therefore stop trying.
By using various properties of Markov chains, we can use this transition matrix $\mathcal P$ to compute the mean time to reach the absorbing state (row 4). We can also find a probability distribution for the time to obtain three consecutive successes, but for this particular matrix, it is a long and convoluted expression. Numeric estimates are provided as follows: $$\begin{array}{c|c|c} x & \Pr[X = x] & \Pr[X \le x] \\ \hline 3 & 0.504 & 0.504 \\ 4 & 0.0504 & 0.5544 \\ 5 & 0.09576 & 0.65016 \\ 6 & 0.127512 & 0.777672 \\ 7 & 0.0408744 & 0.818546 \\ 8 & 0.0477238 & 0.86627 \\ 9 & 0.0396724 & 0.905943 \\ 10 & 0.0213864 & 0.927329 \\ 11 & 0.019588 & 0.946917 \\ 12 & 0.0143776 & 0.961294 \\ 13 & 0.00958306 & 0.970878 \\ 14 & 0.00777728 & 0.978655 \\ 15 & 0.00560823 & 0.984263 \\ 16 & 0.00403067 & 0.988294 \\ 17 & 0.00309244 & 0.991386 \\ 18 & 0.00224614 & 0.993632 \\ 19 & 0.00165188 & 0.995284 \\ 20 & 0.00123746 & 0.996522 \\ \vdots & & \end{array}$$