There are five distinct computer science books, three distinct mathematics books, and two distinct art books. In how many ways can these books be arranged on a shelf if no two of the three mathematics books are together?
I am getting $3507840$ by doing 10! - (3*8!). Is it correct?
Let's count the number of ways we can arrange the books so that there are at least two math books adjacent.
The number of ways we can choose the pair that is adjacent is $3$, and if we include the $2$ ways to order the pair we get $6$ possibilities for the book pair. We can think of the collection of books now as being $9$ books: $7$ non-math books, $1$ math book, and $1$ pair of math books that is inseparable. There are $9!$ ways to arrange these. So we have $$9! \cdot 6$$
But we've overcounted the cases in which all three math books are adjacent. How many such cases are there? Well that's the same as there being $8$ books on the shelf: $7$ non-math books and $1$ giant trilogy of math books, for $8!$ possibilities, times the $3!=6$ ways to arrange the math books within the three-book thing. This gives $$8! \cdot 6$$
Since we've counted each of the three-adjacent cases exactly twice, and we want to count them only once, this means there are $$9! \cdot 6 - 8! \cdot 6$$ ways to arrange the books so that there are at least two math books adjacent.
But we want the number of ways so that they aren't adjacent. So we can subtract from the total number of arrangements, which is $10!$, to get our final answer
$$10! - (9! \cdot 6 - 8! \cdot 6) = \boxed{1693440}$$