I'm studying both Financial Math and Statistics, and it came to my mind how could one interrelate them.
So I thought in an example and started to scribble:
Given two business operation, the first one with 80% of sucess and 5% interest rate per month; the second one with 95% of sucess and 1,5% interest rate per month. Considering that if the operation fails, no money is lost, which one is more likely profitable at medium term (10 months)?
The first one:
$$M = x\cdot(1+0,05)^{10}$$
$$M = 1,63x$$
$$Odds = (\frac{80}{100})^{10} = 10,47\%$$
The second one:
$$M = x\cdot(1+0,015)^{10}$$
$$M = 1,16x$$
$$Odds = (\frac{95}{100})^{10} = 59,87\%$$
In the first operation, the most probable case is 10,47% of earning 63% income; while in the second operation, the most probable case is 59,87% of earning 16% income. But it seems that analysis is not fair, as both have different chances, so I thought that in order to make it pair, I should calculate the odds of the first operation (as it haves more income) to have same income as the second one.
$$(1+0,05)^x = 1,16$$ $$log\;1,05^x = log\;1,16$$ $$x = \frac{log\;1,16}{log\;1,05}$$ $$x = 3,042 \approx 3$$
It has to be a success 3 times to produce the same result, and the odds are:
$$Odds = (\frac{80}{100})^{3} = 51,2\%$$
My questions are:
1 - Is this analysis logical and coherent?
2 - Is it right to assume that the second case is statistically a better choice?
3 - How can this extend into a case that considers money lost at failure?
As noted in David's comment, we cannot say which option is better due to the risk preference of the individual.
One can argue that we should choose the one with the higher expected value.
If we assume that we initially invested $1$ unit of money, the expected value would be $$P(\text{10 months consecutive success}) \cdot (1 + \text{monthly interest rate})^{10} + P(\text{fail somewhere in the 10 months}) \cdot 1.$$
For the first option, we get $$0.8^{10} \cdot (1 + 0.05)^{10} + (1 - 0.8^{10}) \cdot 1 \approx 1.0675,$$ while for the second option, we get $$0.95^{10} \cdot (1 + 0.015)^{10} + (1 - 0.95^{10}) \cdot 1 \approx 1.0961.$$
With that, you can argue that the second option is better.
However, one can argue that the above is not reasonable.
Consider the following scenario. I offer you two options:
Realistically speaking, which will you choose?
If we compute the expected values, the first option gives $$100\% \cdot 10^9 = 10^9,$$ while the second option gives $$50\% \cdot 10^{10} + 50\% \cdot 1 = 5 \cdot 10^9 + 0.5.$$
So, it seems the second option is better.
Yet, I believe that most people will choose the first option. Even though $10^9$ USD is smaller, most people can live comfortably with $10^9$ USD in their bank account. So, it's reasonable to take the guaranteed $10^9$ USD rather than the uncertain $10^{10}$ USD.
This is where utility function comes into play. Instead of computing the expected value, we instead compute the expected utility value. Roughly speaking, you define the 'worthiness' of a money using a utility function $U(x)$, where $x$ is the amount of money.
One common example is $U(x) = \log x$. This is a reasonable choice, since one can argue the worthiness of money follows a logarithmic scale, i.e., it diminishes in value. For instance, give a billionaire and a poor person one million USD each; the poor person will likely be much happier than the billionaire.
If we set $U(x) = \log_{10} x$, the expected utility value of the second option becomes $$50\% \cdot U(10^{10}) + 50\% \cdot U(1) = 50\% \cdot 10 + 50\% \cdot 0 = 5,$$ while the expected utility value of the first option becomes $$100\% \cdot U(10^9) = 9.$$ So, the first option now seems better!
Coming back to your original question, my answer is: it depends on the utility function.
Different individuals have different utility functions, and it affects which choice is better for those individuals. We see this a lot in real life, e.g., some people play risky and invest in stocks, while others just place their money in a savings account with a smaller interest rate but less risk. This is particularly similar to your question too.
Is it better to choose a relatively high risk option that has high reward, or choose the one that is less risky but less rewarding? It depends.
Disclaimer: I am not saying that this expected utility concept is the 'best' way to judge the worthiness of a resource. This is just simply one common way to do so — at least, in my basic financial mathematics classes.
P.S. We can try finding the expected utility of the two original options you gave using several utility functions. For example, with $U(x) = \log_{10}(x)$, the second option is still better, while with $U(x) = x^3$, the first option is better.
P.P.S. The function $U(x) = \log_{10}(x)$ isn't exactly a good choice for a utility function, since it's undefined when $x$ is not positive. I chose this function because it's relatively easy to use for the computations.
Edit:
The answer above assumed you will lose all interest from all months when you fail once. If you only lose the current month's interest, then the expected value would be different.
If $p$ is the probability of a success for a month, then the probability of exactly $k$ successes among $10$ months is $\binom{10}{k} p^k (1-p)^{10-k}$. With these number of successes, the investment gives you $I \cdot (1+r)^k$, where $I$ is the initial investment value and $r$ is the monthly interest rate.
Hence, the expected value would be $$\sum_{k=0}^{10} I(1+r)^k \binom{10}{k} p^k (1-p)^{10-k}.$$ With this interpretation, if we set $I = 100$, the first and the second option gives approximately $148.02443$ and $115.19939$ respectively.