So, a friend of mine wants me to write them a program to calculate certain probabilities for their tabletop RPG. The mechanic is essentially, a player has a pool of $10$-sided dice, which they roll, and any result $5$ or above is a success, any result less than $5$ is a failure, and there is a a set number of total successful rolls they need to succeed overall. This feels fairly straightforward, here's the function I'm using to calculate that: $$ \sum_{i=c}^{p}\binom{p}{i}5^i\cdot(10-5)^{(p-i)} $$ where $p$ is the number of dice, $c$ is the "challenge", the number of required successful rolls to succeed overall, $5$ is the number of ways a roll can suceed, and $(10-5)$ is the number of ways it can fail. However this is insufficient because there are also special rules and mechanics dependent on rolling either $10$ or $1$ that I need to account for. My solution so far is just to handle them manually, i.e. sum over (for $5$ dice):
1 1 1 1 1 ? ? ? ? ? 10
1 1 1 1 ? ? ? ? ? 10 10
1 1 1 ? ? ? ? ? 10 10 10
1 1 ? ? ? ? ? 10 10 10 10
1 ? ? ? ? ? 10 10 10 10 10
where ? is an unknown quantity which is now essentially an 8-sided die, which conveniently still carries the same probability. So, my question: I have both of these components, but I'm having a very difficult time wrapping my brain around how to modify the distribution function above to take into account the known quantity of $10$s and $1$s (which are successes and failures respectively). Does it modify the probability? Does it multiply the result by a constant to the power of $i$? Or something else? Thanks!
"One-hot" representation
If you are starting from scratch, probably the easiest solution is to consider the die as a multivariate distribution where a single die produces either a 0 or 1 on each axis. In this case, you would have a 3-vector (botch, success, crit) with the following outcomes:
Then you can find the total number of botches, successes, and crits by taking successive sum distributions of 1 + 1 dice, 2 + 1 dice, 3 + 1 dice, etc.
Multinomials
Another option is to use the multinomial distribution directly to get the number of botches, failures, successes, and crits. This is probably the most popular approach in practice, being used by Troll and AnyDice.
Binomial decomposition of multinomials
However, your original approach does have merit---it is possible to decompose the multinomial coefficient into a product of binomial coefficients. In fact, this leads to efficient dynamic programming algorithms for dice pools, though perhaps overkill for this particular problem. If you're curious, you can read my paper on the subject.
More to the point, this sounds like the mechanic used in Vampire: The Masquerade 5th Edition. You can try these approaches applied to V5 in your browser using this JupyterLite notebook, which uses my own Icepool Python library.