Suppose I want to choose $12$ chocolates from $3$ different varieties:
- $10$ milk chocolates,
- $10$ dark chocolates, and
- $1$ fat-free chocolate
In how many ways can I do this? I tried $$\frac{_{21}P_{12}}{10!\cdot10!}=\frac{21!}{9!\cdot10!\cdot10!}$$ ($10!\cdot10!$ account for repetitions)
which apparently isn't correct.
Let $m$, $d$, and $f$ be, respectively, the number of milk, dark, and fat-free chocolates. Then we want to find the number of solutions of the equation $$m + d + f = 12$$ in the nonnegative integers subject to the constraints that $m \leq 10$, $d \leq 10$, and $f \leq 1$.
The constraint $f \leq 1$ is best handled, as @user2661923 suggested in the comments, by considering two cases, depending on whether the selection includes a fat-free chocolate.
Case 1: The fat-free chocolate is not selected.
Since the fat-free chocolate is not selected, $f = 0$, so we must find the number of solutions of the equation $$m + d = 12$$ in the nonnegative integers subject to the constraints $m \leq 10$, $d \leq 10$. Notice that choosing how many milk chocolates are selected determines how many dark chocolates are selected. Since $d \leq 10$, at least how many milk chocolates must be selected? At most how many milk chocolates may be selected?
Case 2: The fat-free chocolate is selected.
Since the fat-free chocolate is selected, $f = 1$, so we must find the number of solutions of the equation $$m + d = 11$$ in the nonnegative integers subject to the constraints $m \leq 10$, $d \leq 10$. Again, choosing how many milk chocolates are selected determines how many dark chocolates are selected. Since $d \leq 10$, at least how many milk chocolates must be selected? At most how many milk chocolates may be selected?
Total: The two cases are mutually exclusive and exhaustive, so the total can be found by adding the results of the two cases.
What was wrong with your approach?
You selected $12$ items but divided by $10!10!$. However, $10 + 10 = 20 > 12$. The number of distinguishable ways six milk chocolates, five dark chocolates, and one fat-free chocolate can be selected is $$\binom{12}{6}\binom{6}{5}\binom{1}{1} = \frac{12!}{6!6!} \cdot \frac{6!}{5!1!} \cdot \frac{1!}{1!0!} = \frac{12!}{6!5!1!}$$ where we choose six positions in the sequence of twelve selections for the milk chocolates, five of the remaining six positions for the dark chocolates, and fill the remaining position with the fat-free chocolate. Notice that $6 + 5 + 1 = 12$.
Since you would have to list all the possible selections in order to do the problem in this way, it is not an efficient approach to the problem of counting how many selections there are.