I tried to solve this question by using the following formula for permutation with repeating values (source: https://www.ck12.org/probability/permutations-with-repetition/lesson/Permutations-with-Repetition-BSC-PST/ ):
$$
\frac{{n}P{k}}{x_1!x_2!x_3!}
$$
Since there are 3 letters each repeating 3 times in the set, and we’re selecting 3 letters out of the set of 9 letters, I ended up with the following result:
$$ \frac{9!}{(9-3)!3!3!3!}=\frac{9\cdot8\cdot7}{3!3!3!}=\frac{504}{216}=2.333 $$
I’m pretty sure $2.333$ isn’t the right answer, considering how ABC, AAA, BBB, CCC already form four permutations, and I don’t think there can be $2.33$ permutations. What am I doing wrong?
There are three ways to choose each of three letters. Therefore the total number of "permutations" is $$3^3=27.$$
UPDATE:
In general the number of ordered $k$-subsets of a multiset containing $n_i$ elements of type $i$ ($i=1\dots M$) is the coefficient at $x^k$ of the polynomial: $$ k! \prod_{i=1}^M\sum_{j=0}^{n_i}\frac{x^j}{j!}\tag1 $$
In your example it is: $$ \left[3!\left(1+\frac x{1!}+\frac{x^2}{2!}+\frac{x^3}{3!}\right)^3\right]_3=27. $$