Find the number of ways in which 6 persons out of 5 men and 5 women can be seated at a round table such that 2 men are never together.
My attempt:
6 people may be 3 men and 3 women, 2 men and 4 women or 1 man and 5 women.
Then by the gap method, 3 men and 3 women can be seated in $\binom{5}{3}2!3!=120$ ways.
2 men and 4 women can be seated in $\binom{5}{4}3!\frac{4!}{2!}=360$ ways
1 men and 5 women can be seated in $\binom{5}{5}4!\frac{5!}{4!}=120$ ways
So the total number of ways is 600, but the answer given in my book is 5400. Where did I go wrong?
You forgot to include the number of ways in which the men can be selected. In this case, you get:
$${5 \choose 3}{5 \choose 3}2!3! = 1200$$ $${5 \choose 2}{5 \choose 4}3!\frac{4!}{2!} = 3600$$ $${5 \choose 1}{5 \choose 5}4!\frac{5!}{4!} = 600$$
This results in a total of $1200 + 3600 + 600 = 5400$ possibilities.